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PREFACE 


This  paper  was  prepared  by  the  Institute  for  Defense  Analyses  (IDA)  for  the  Office 
of  the  Director,  Acquisition  Policy  and  Program  Integration  under  a  task  entitled  “Indicators 
of  Cost  and  Schedule  Growth.”  The  objective  of  the  task  is  to  describe  cost  and  schedule 
growth  patterns  associated  with  the  acquisition  of  selected  major  systems,  identify  reasons 
for  the  growth,  and  develop  a  way  to  predict  growth  in  ongoing  development  and  early 
production  phases.  This  paper  examines  cost  and  schedule  growth  in  tactical  missile  and 
tactical  aircraft  programs. 

This  work  was  reviewed  within  IDA  by  Stanley  A.  Horowitz,  Thomas  P.  Frazier, 
and  David  R.  Graham. 
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EXECUTIVE  SUMMARY 


A.  BACKGROUND 

Programs  to  acquire  major  weapon  systems  often  end  up  taking  longer  and  costing 
more  than  planned.  The  time  requited  to  design  and  develop  major  weapon  systems  has 
apparently  lengthened,  and  weapon  systems  often  exceed  their  planned  development 
schedules. 

Excessive  schedules  have  two  significant  negative  effects:  U.S.  forces  may  be  left 
without  needed  capabilities  and  longer  schedules  often  mean  higher  costs.  Growth  in  the 
cost  of  weapon  systems  appears  to  be  a  chronic  problem:  66  out  of  82  programs  examined 
by  EDA  in  1992  had  experienced  cost  growth. 

The  defense  acquisition  community  is  concerned  that  programs  to  develop  major 
weapon  systems  take  too  long  and  cost  too  much.  Cost  growth  forces  the  DoD  to  revise 
budget  plans,  makes  systems  less  affordable,  and  frequently  erodes  congressional  support 
for  acquisition  programs.  Given  recent  reductions  in  the  money  available  for  defense,  the 
acquisition  community  needs  a  better  tinderstanding  of  the  causes  of  cost  and  schedule 
growth,  so  that  those  plans  can  be  made  more  realistic. 

B .  OBJECTIVE  AND  APPROACH 

DoD  asked  IDA  to  describe  cost  and  schedule  growth  patterns  and  to  identify 
reasons  for  the  growth.  By  looking  at  past  programs,  we  examined  hypotheses  about  what 
separates  the  kinds  of  programs  completed  on  schedule  and  within  cost  plans  from  those 
that  suffer  from  schedule  and  cost  growth. 

The  study  focused  on  tactical  missiles  and  tactical  aircraft.  We  developed  both 
qualitative  and  quantitative  relationships  between  cost  growth  and  schedule  growth  in 
missile  programs.  In  addition,  we  developed  quantitative  cost/schedule  relationships  for  a 
small  sample  of  tactical  aircraft. 

C.  REASONS  FOR  COST  AND  SCHEDULE  GROWTH 

Cost  and  schedule  growth  for  the  tactical  missiles  in  our  sample  were  measured  in 
development  and  in  production.  These  measures  showed  a  great  deal  of  variability  among 
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the  twenty  progranis  examined.  Programs  took  from  50  months  to  137  months  from 
Milestone  n  to  inital  operational  capability.  Only  two  of  the  tactical  missile  programs  were 
finished  on  time.  The  program  with  the  highest  development  schedule  growth  exceeded  its 
plan  by  180  percent.  Two  programs  were  completed  under  budget,  while  two  others  cost 
more  than  double  their  Milestone  n  plan. 

Selected  tactical  missile  programs  were  examined  in  more  detail  to  determine  the 
reasons  for  schedule  and  cost  growth.  Table  S-I  summarizes  the  characteristics  of 
programs  with  low  and  high  levels  of  development  schedule  growth.  Keys  to  preventing 
schedule  growth  in  development  are  technical  realism  and  willingness  to  make  tradeoffs. 
Programs  with  high  development  schedule  growth  tended  to  underestimate  technical 
difficulty.  Two  of  the  five  programs  with  high  development  schedule  growth  also  had  high 
overall  cost  growth.  However,  in  three  of  the  five  cases  of  high  development  schedule 
growth,  a  strictly  phased  approach  (resolving  problems  in  development  when  spending 
levels  are  low)  appeared  to  result  in  lower  levels  of  overall  cost  growth. 

Table  S-2  shows  characteristics  of  missile  programs  with  low  or  high  total  program 
cost  growth.  Keys  to  preventing  overall  cost  growth  are  correctly  estimating  the  degree  of 
technical  difficulty  in  the  programs  and  maintaining  the  planned  production  schedule. 
Programs  that  employed  a  high  degree  of  concurrency,  that  had  to  be  dual-sourced  for 
technical  reasons  or  that  were  dual-sourced  at  less  than  full  rate,  had  high  cost  growth.  In 
one  case,  the  threat  of  competition  appeared  to  reduce  costs. 

Cost  and  schedule  growth  measures  were  also  calculated  for  a  sample  of  seven 
tactical  aircraft.  These  measures  are  less  dispersed  than  those  for  tactical  missiles.  The 
aircraft  programs  tended  to  receive  more  management  attention  and  more  protection  from 
schedule  stretch  than  the  tactical  missiles.  The  highest  cost  growth  index  for  tactical  aircraft 
was  1.40,  versus  2.23  for  the  tactical  missiles.  In  two  cases  (the  F-14A  and  the  AV-8B), 
programs  were  stretched  in  production  but  did  not  suffer  from  the  high  cost  growth  seen  in 
the  missile  programs.  In  the  case  of  the  F-14A.  this  could  have  been  due  to  a  combination 
of  early  warning  and  the  presence  of  the  F-14D  development  program  to  cushion  the  blow. 
When  the  AV-8B  was  stretched  out,  the  fact  that  the  F/A-18  and  F-15  programs  were  using 
the  same  plant  may  have  helped  to  spread  overhead  costs  and  contain  cost  growth  in  the 
AV-8B  program.  The  program  with  the  highest  production  cost  growth  is  the  F/A-18, 
which  exhibited  production  cost  growth  of  42  percent.  Technical  changes  made  late  in  the 
process  contributed  to  its  high  cost  growth. 
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T«bl»  S-2.  Characteristics  of  Programs  With  Low  and  High  Cost  Growth  in  Total  Program 
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D.  RELATIONSHIP  BETWEEN  SCHEDULE  GROWTH  AND  COST 
GROWTH 

We  also  considered  the  relationship  between  schedule  growth  and  cost  growth. 
Does  schedule  growth  necessarily  lead  to  cost  growth,  or  do  other  factors  intervene?  Is  the 
relationship  between  cost  growth  and  schedule  growth  different  for  different  equipment 
types,  or  for  different  phases  of  the  acquisition  cycle? 

In  addition  to  examining  the  relationship  between  cost  and  schedule,  we  considered 
whether  other  variables  affect  cost  growth: 

•  Are  programs  to  develop  completely  new  systems  more  prone  to  cost  growth 
than  programs  to  modify  existing  systems? 

•  Are  “crash”  programs  inherently  more  susceptible  to  cost  growth?  Does 
concurrency  (simultaneous  or  overlapping  development  and  production) 
necessarily  doom  a  program  to  high  cost  growth? 

•  How  much  does  adherence  to  the  planned  production  schedule  influence  cost 
growth  in  production?  Do  programs  whose  procurement  is  stretched  out  have 
higher  costs? 

In  the  case  of  tactical  aircraft,  the  one  program  that  was  a  modification  of  an 
existing  system — the  AV-8B — had  low  cost  growth.  However,  tactical  missile 
modifications  frequently  have  very  high  cost  growth.  This  may  be  because  modifications 
are  often  made  to  the  guidance  and  control  system,  the  most  expensive  part  of  the  missile. 

Programs  with  time  urgency  had  both  high  and  low  cost  growth.  Those  with  low 
cost  growth  used  relatively  simple  technologies  with  careful  testing  and  avoided  production 
stretch.  Concurrency  was  positively  related  to  production  cost  growth,  although  the 
significance  level  was  marginal  (.12). 

As  indicated  by  the  aircraft,  production  stretch  need  not  necessarily  lead  to  higher 
cost  growth.  In  the  cases  of  the  F-14A  and  the  AV-8B.  plenty  of  warning  and  the  ability  to 
reallocate  resources  to  other  programs  in  the  same  plant  helped  to  lessen  cost  growth. 
Without  these  mitigating  factors,  though,  production  schedule  stretch  usually  leads  to  cost 
growth. 

There  were  enough  common  factors  in  the  tactical  missiles  to  suggest  that  the 
estimation  of  quantitative  relationship.^  would  be  possible.  The  equations  presented  in  this 
report  relate  cost  growth  to  schedule  growth  in  development  and  production.  In 
development,  a  simultaneous  model  links  an  estimating  relationship  for  schedule  growth 
with  a  cost  growth  equation.  The  major  determinant  of  development  schedule  growth  was 
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increase  in  quantity — the  need  to  produce  more  items  for  testing  than  planned.  Other 
variables  in  the  DSG  equation  were  the  planned  schedule  for  the  program  and  dummy 
variables  for  intercept  missiles  and  for  one  outlier.  The  results  of  this  equation  feed  into  a 
development  cost  growth  equation.  We  also  developed  a  simpler,  single-equation 
alternative  relating  development  cost  growth  to  development  schedule  growth  and 
development  quantity  growth.  Cost  growth  in  missile  production  was  linked  to  schedule 
stretch,  planned  unit  cost  (a  proxy  for  complexity),  and  multiyear  procurement.  Total 
program  cost  growth  was  related  to  total  schedule  growth,  planned  unit  cost,  and  an 
intercept  missile  dummy  variable.  The  total  program  cost  growth  equation  is: 

TPCG  =  .7645  +  (.3677  x  TSG)  +  (.1845  x  PUC)  +  (.2729  x  IMD) 

(.005)  (.04)  (.04) 

Adjusted  =  .500  SEE  =  259 

where  TPCG  is  total  program  cost  growth,  TSG  is  total  schedule  growth,  PUC  is  planned 
unit  cost  in  millions  of  1994  dollars,  and  IMD  is  set  equal  to  1  for  intercept  missiles  and  0 
otherwise.  Figure  S-1  shows  the  fit  of  the  total  program  cost  growth  equation  for  tactical 
missiles. 


Figure  S-l.  Total  Program  Coat  Growth  for  Tactical  Miasilea 
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Quantitative  relationships  were  also  developed  for  the  aircraft,  but  the  small  sample 
size  requires  that  they  be  regarded  as  tentative.  !n  production  and  the  total  program,  a 
logarithmic  form  using  schedule  length  rather  than  schedule  growth  provided  the  best  fit. 
The  arithmetic  form  of  the  aircraft  total  program  equation  is: 

TPCG  =  .3785  X  ATS  -2365  EAV8B  -3962 
(.003)  (.006) 

Adjusted  =  .890  SEE  =  .053 

where  TPCG  is  total  program  cost  growth,  ATS  is  actual  total  schedule,  and  EAV8B  takes 
the  value  e  for  the  AV-8B  and  1  for  all  other  aircraft.  Numbers  in  parentheses  below  the 
coefficients  are  significance  levels.  A  scatter  plot  of  TPCG  and  ATS  illustrating  the  fit  of 
the  equation  is  shown  in  Figure  S-2. 


Figure  S-2.  Total  Program  Cost  Growth  for  Tactical  Aircraft 


The  tactical  missile  regressions  generally  fit  well  and  can  reasonably  be  used  for 
getting  a  sense  of  the  cost  implications  of  schedule  changes.  Due  to  the  small  sample  size, 
additional  development  work  is  needed  to  make  the  aircraft  regressions  into  practical 
projection  tools.  This  exercise  indicated  that  the  cost/schedule  relationships 
different  for  different  equipment  types. 

E.  IMPLICATIONS  FOR  DOD 

In  the  light  of  these  results,  we  conclude  that  DoD  and  other  personnel  who  review 
acquisition  programs  would  benefit  from  a  review  method  based  on  detailed  information 
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about  the  strategies  and  outcomes  of  past  programs.  This  is  the  first  step  for  development 
of  such  a  review  method  that  examines  the  reasonableness  of  program  plans  and  assesses 
the  cost  impact  of  schedule  changes.  The  equations  in  this  report  can  be  used  to  identify 
programs  likely  to  experience  future  growth,  as  well  as  to  monitor  the  effect  of  schedule 
changes  on  cost  growth.  They  can  be  used  to  identify  which  programs  require  more 
detailed  examination. 

The  case  analyses  of  tactical  missiles  also  indicate  that  there  are  many  useful  lessons 
to  be  gleaned  from  historical  perspective,  lessons  that  cannot  always  be  captured  in  a 
quantitative  estimating  relationship.  For  the  tactical  aircraft,  the  1990s  will  be  a  decade 
where  only  a  few  prime  contractors  will  survive.  If  programs  are  stretched  out,  contractors 
are  not  likely  to  be  able  to  cushion  the  blow  by  working  on  other  programs. 

At  the  beginning  of  a  weapon  system  development  program,  the  feeling  prevails 
that  that  program  will  avoid  repeating  the  problems  of  previous  programs.  Yet  cost  and 
schedule  growth  persist.  Despite  individual  differences  in  programs,  the  importance  of 
understanding  the  level  of  technical  difficulty  when  original  schedule  and  cost  estimates  are 
made,  of  strict  phasing  and  vigorous  testing,  and  of  adhering  to  production  plans  are  borne 
out  by  analysis  of  past  strategies  and  outcomes. 
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I.  INTRODUCTION 


A.  BACKGROUND 

Programs  to  acquire  major  weapon  systems  usually  cannot  follow  their  carefully- 
laid  acquisition  plans  in  every  detail.  They  often  end  up  taking  longer  and  costing  more 
than  planned. 

The  time  required  to  design  and  develop  major  weapon  systems  has  apparently 
lengthened,  and  weapon  systems  often  exceed  their  planned  development  schedules.  A 
1990  RAND  Corporation  study  found  that  programs  from  the  1970s  and  1980s  took  longer 
than  those  from  the  1950s  and  1960s.  The  demonstration/validation  phase  (from  Milestone 
I  to  Milestone  11  in  the  process  required  by  DoD  regulations)  and  the  engineering  and 
manufacturing  development  phase  (measured  from  Milestone  11  to  the  first  delivery)  each 
talce  about  a  year  longer  than  they  used  to  [1].  A  1992  IDA  study  found  that,  on  average, 
systems  take  one-third  longer  than  planned  to  progress  from  Milestone  n  to  achievement  of 
initial  operational  capability  (ICXI!)  [2]. 

Excessive  schedules  have  two  significant  negative  effects.  One  is  that  delays  in 
schedules  may  leave  U.S.  forces  without  needed  capabilities  and  vulnerable  to  enemy 
weapons,  if  the  system  is  not  completed  on  time.  The  other  is  that  delays  often  translate 
into  higher  cost,  which  creates  further  problems. 

Weapon  system  costs  have  been  growing  both  from  generation  to  generation,  and, 
within  a  given  system,  from  plan  to  realization.  Growth  in  the  cost  of  weapon  systems 
appears  to  be  a  chronic  problem.  Of  82  programs  examined  by  IDA  in  1992,  66  had 
experienced  cost  growth  [2]. 

The  defense  acquisition  community  is  concerned  that  programs  to  develop  major 
weapon  systems  take  too  long  and  cost  too  much.  In  1986,  the  President’s  Blue  Ribbon 
Conunission  on  Defense  Management  (the  “Packard  Commission")  called  schedule  length 
“a  central  problem  from  which  most  other  acquisition  problems  stem”  [3].  Cost  growth 
forces  the  DoD  to  revise  budget  plans,  makes  systems  less  affordable,  and  frequently 
erodes  congressional  support  for  acquisition  programs.  Recent  reductions  in  the  money 
available  for  defense  mean  that  the  acquisition  conununity  needs  to  increase  understanding 
of  the  reasons  for  deviations  from  progi-am  schedule  and  cost  plans. 
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The  large  variations  in  total  development  time,  in  transition  from  development  to 
production,  and  in  schedule  growth  suggest  that  it  is  possible  to  reduce  development  times 
and  to  improve  early  estimates  of  schedules.  Similarly,  the  large  variation  in  cost  growth — 
from  little  or  no  cost  growth  to  more  than  double  planned  costs — suggests  that  potential 
exists  for  estimating  costs  more  accurately.  With  a  better  understanding  of  the  causes  of 
cost  and  schedule  growth,  DoD  can  review  program  plans  early  in  the  process  and  make 
them  more  realistic. 

B .  PURPOSE  AND  APPROACH 

IDA  was  asked  by  DoD  to  describe  cost  and  schedule  growth  patterns  and  to 
identify  reasons  for  the  growth.  By  looking  at  past  programs,  we  examined  hypotheses 
about  what  separates  the  kinds  of  programs  completed  on  schedule  and  within  cost  plans 
from  those  that  suffer  from  schedule  and  cost  growth.  We  also  considered  the  relationship 
between  schedule  growth  and  cost  growth.  Does  schedule  grov/th  necessarily  lead  to  cost 
growth,  or  do  other  factors  intervene?  Is  the  relationship  between  cost  growth  and  schedule 
growth  different  for  different  equipment  types,  or  for  different  phases  of  the  acquisition 
cycle? 

In  addition  to  examining  the  relationship  between  cost  and  schedule,  we  considered 
whether  other  variables  affect  cost  growth: 

•  Are  programs  to  develop  completely  new  systems  more  prone  to  cost  growth 
than  programs  to  modify  existing  systems? 

•  Are  “crash”  programs  inherently  more  susceptible  to  cost  growth?  Does 
concurrency  (simultaneous  or  overlapping  development  and  production) 
necessarily  doom  a  program  to  high  cost  growth? 

•  How  much  does  adherence  to  the  planned  production  schedule  influence  cost 
growth  in  production?  Do  programs  whose  procurement  is  stretched  out  have 
higher  costs? 

We  focused  on  cost  and  schedule  growth  in  two  major  product  categories,  tactical 
missiles  and  tactical  aircraft.  Tactical  missiles  are  particularly  prone  to  cost  and  schedule 
difficulties,  as  prior  IDA  research  has  shown  [2,  4,  and  5].  Tactical  missiles,  while 
technologically  complex,  are  viewed  as  being  less  “glamorous”  than  tactical  aircraft,  and 
missile  programs  often  do  not  receive  priority.  Tactical  aircraft  are  among  the  most 
technologically  complex  products  the  military  buys.  They  are  developed  on  a  custom  basis, 
usually  pushing  the  state  of  the  art.  Tactical  aircraft  development  is  constrained  by  size  and 
weight  restrictions,  and  is  spurred  toward  ever-faster  goals  for  operating  speeds. 
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Moreover,  the  physical  environment  in  which  these  aircraft  operate  often  involves  extreme 
temperatures,  high  vibration,  and  G-forces  [6]. 

We  developed  both  qualitative  and  quantitative  relationships  between  cost  and 
schedule  in  missile  programs.  In  addition,  uc  developed  quantitative  cost/schedule 
relationships  for  a  small  sample  of  tactical  uircrutt. 

C.  ORGANIZATION  OF  THIS  REPORT 

Chapter  II  of  this  report  discusse.s  the  data  used,  including  definitions  of  the  key 
program  outcome  measures,  and  includes  tables  of  the  basic  data.  Chapter  III  discusses 
detailed  reasons  for  cost  and  schedule  growth,  based  on  case  analyses  of  15  missile 
programs.  Chapter  IV  provides  quantitative  relationships  between  cost  growth  and 
schedule  growth.  Chapter  V  summarizes  conclusions  from  the  qualitative  and  quantitative 
analyses  and  discusses  the  implications  for  DoD. 
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II.  DATA 


A.  INTRODUCTION 

For  this  study,  IDA  examined  twenty  tactical  missiles  and  seven  tactical  aircraft. 
Among  them  were  programs  classiHed  as  major  systems.  None  of  the  programs  in  our 
sample  were  canceled.  The  data  on  these  systems  were  obtained  from  Selected  Acquisition 
Reports  (SARs),  from  historical  memoranda  to  support  DoD  program  reviews,  and  from 
summaries  of  program  data  [7  and  8].  Data  were  current  as  of  the  1992  SARs. 

We  examined  only  the  first  version  of  each  system  and  not  subsequent 
modifications,  unless  they  were  considered  as  separate  programs.  DoD  treatment  of 
modification  programs  in  its  reporting  and  review  process  has  not  always  been  uniform. 
Sometimes,  as  in  the  case  of  the  F-14,  a  new  version  was  treated  as  a  new  program  and 
went  through  a  new  set  of  reviews  and  had  separate  documentation.  Other  times,  as  in  the 
case  of  the  F-15,  DoD  treated  the  program  as  one,  despite  the  technical  differences  between 
the  F-15A/B  and  the  F-15E.  When  a  data  source  included  costs  for  modified  versions,  we 
used  data  on  production  schedules  to  obtain  the  actual  costs  for  the  first  version. 


B .  MEASUREMENT  OF  COST  AND  SCHEDULE  GROWTH 

Major  weapon  system  programs  undergo  systematic  reviews  at  key  points  in  the 
process.  These  so-called  milestone  reviews  are  designed  to  measure  the  program’s 
progress  against  its  goals  in  all  phases  of  the  process.  The  major  phases  are: 

•  Phase  0:  Concept  Exploration  and  Definition 

•  Phase  I:  Demonstration  and  Validation 

•  Phase  0:  Engineering  and  Manufacturing  Development 

•  I%ase  ni;  Production  and  Deployment 

•  Phase  IV:  Operations  and  Support 


An  expert  panel,  the  Defense  Acquisition  Board  (DAB),  judges  each  program’s 
progress  relative  to  the  exit  criteria  for  each  phase  and  issues  decisions  about  the  future 
course  of  each  program.  The  board  is  chaired  by  the  Under  Secretary  of  Defense 


(Acquisition  and  Technology).  This  panel  was  formerly  called  the  Defense  Systems 
Acquisition  Review  Council  (DSARC). 

Figure  11- 1  provides  a  schematic  of  the  milestone  process.  At  Milestone  II,  the 
government  describes  the  system  and  makes  baseline  estimates  of  cost  and  schedule.  If  the 
system  receives  DAB  approval,  it  moves  into  engineering  and  manufacturing  development. 
(For  much  of  the  time  represented  by  this  study,  this  phase  was  called  full-scale 
development  or  FSD.)  The  cost  and  schedule  estimates  are  updated  annually  to  reflect 
experience  and  greater  knowledge  about  the  system. 

We  examined  cost  and  schedule  growth  by  comparing  actual  outcomes  with  those 
planned  at  the  Milestone  II  review,  when  the  commitment  for  engineering  and 
manufacturing  development  was  made.  Outcomes  were  expressed  as  a  ratio  of  actual  to 
planned  cost  or  schedule. 

The  development  cost  growth  (DCG)  ratio  is  defined  as; 

DCG  =  ADC  +  PDC, 

where 

ADC  =  actual  development  cost,  the  actual  cost  to  develop  the  system, 
measured  in  millions  of  FY  1994  dollars  from  Milestone  II  to  the  end 
of  development  of  the  first  version;  and 

PEXZ  =  planned  development  cost,  the  planned  cost  to  develop  the  system, 
measured  in  millions  of  FY  1994  dollars  from  Milestone  II  to  the  end 
of  development  of  the  first  version. 

The  production  cost  growth  (PCG)  ratio  is  defined  as: 

PCG  =  APC  +  PPC, 

where 

APC  =  actual  production  cost,  the  actual  cost  to  produce  the  planned  quantity 
of  the  system,  measured  in  millions  of  FY  1994  dollars;  and 

PPC  =s  planned  production  cost;  the  planned  cost  at  Milestone  II  to  produce  the 
planned  quantity  of  the  system,  measured  in  millions  of  FY  1994 
dollars. 

If  the  actual  quantity  produced  was  greater  than  that  planned  at  Milestone  II,  the  numerator 
was  the  cost  to  produce  only  the  planned  quantity.  If  the  actual  quantity  was  less  than  that 
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planned,  wc  used  a  price-improvement  curve  to  project  the  cost  of  producing  the  planned 
quantity.  In  every  case,  we  included  data  only  on  actual  costs.  We  did  not  use  DoD  future 
planned  costs  to  measure  cost  growth. 

The  total  program  cost  growth  (TPCG)  ratio  is  defined  as: 

TPCG  =  (ADC  +  APC)+  (PDC  +  PPC). 

Schedule  measures  were  calculated  similarly.  The  development  schedule  growth 
(DSG)  ratio  is  defined  as: 

DSG  =  ADS  +  PDS, 

where 

ADS  =  actual  development  schedule,  actual  time  to  develop  the  first  version  of 
the  system,  measured  in  months  from  Milestone  11  to  initial  operational 
capability  (IOC);  and 

PDS  s  planned  development  schedule,  planned  time  to  develop  the  first 
version  of  the  system,  measured  in  months  from  Milestone  H  to  IOC. 

The  production  schedule  stretch  (PSS)  ratio  is  defined  as: 

PSS  s  APS  +  PPS, 

where 

APS  s  actual  production  schedule,  actual  time  to  produce  the  planned  quantity 
of  the  system,  measured  in  months  from  Milestone  III  to  the  end  of 
production  of  the  planned  quantity;  and 

PPS  =  planned  production  schedule,  planned  time  to  produce  the  plann''d 
quantity  of  the  system,  measured  in  months  from  Milestone  III  to  the 
end  of  production  of  die  planned  quantity. 

The  timing  of  production  is  based  on  obligation  of  funds,  not  deliveries,  because  that  is  the 
way  the  plans  are  made.  If  the  planned  quantity  has  not  yet  been  produced,  a  linear 
projection  was  made  based  on  actuals. 

Total  schedule  growth  (TSG)  ratio  is  defined  as  follows: 

TSG  =  (ADS+APS)  1-  (PDS+PPS). 

TPCG  and  TSG  may  be  useful  in  understanding  the  effect  of  development 
disruptions  or  successes  on  the  total  program.  For  example,  we  may  observe  that  a 
program  completed  development  seriously  behind  schedule,  but  within  its  planned  cost. 
While  it  is  possible  at  that  juncture  to  have  a  production  program  proceed  as  planned,  it  is 


also  possible  that  issues  of  performance,  producibility,  or  maintainability  are  not 
completely  resolved  until  after  production  is  underway,  adding  to  production  costs. 


The  definitions  of  cost  and  schedule  growth  in  this  paper  differ  from  those  in  past 
IDA  studies  [2  and  5].  In  those  studies,  we  included  all  development  spending  in  our  cost 
growth  measure,  while  in  the  current  study,  we  used  only  spending  past  Milestone  II. 
Production  schedule  stretch  is  a  new  measure  that  captures  the  time  to  produce  the  planned 
quantity.  These  revisions  provide  measures  that  are  appropriate  to  the  issues  raised  in  the 
current  study. 

C.  TACTICAL  MISSILES 

Table  II- 1  presents  schedule  data  for  the  sample  of  twenty  tactical  missiles,  and 
Table  II-2  presents  cost  and  schedule  growth  measures.  As  seen  in  Table  II- 1,  the 
programs  for  which  information  could  be  obtained  represent  a  span  of  twenty  years,  with 
FSD  start  dates  ranging  from  1962  to  1982.  The  sample  includes  tactical  missiles  from  the 
three  services  and  includes  both  air-launched  and  surface-launched  missiles. 

D.  TACTICAL  AIRCRAFT 

Table  II-3  presents  schedule  data  for  the  sample  of  seven  tactical  aircraft,  and 
Table  IM  presents  cost  and  schedule  growth  measures.  The  sample  included  the  major 
fighter  aircraft  programs  over  the  past  three  decades.  The  F-14D  was  omitted,  because  its 
experience  was  much  more  limited  than  the  others,  and  in  1992,  it  was  on  the  verge  of 
cancellation. 
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Table  11-1.  Schedule  Data  for  Tactical  Missiles 
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Tabto  11-2.  Cost  and  Schedule  Growth  Measures  for  Tactical  Missiles 
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m.  REASONS  FOR  COST  AND  SCHEDULE  GROWTH 


A.  INTRODUCTION 

In  this  chapter,  we  discuss  the  reasons  for  cost  and  schedule  growth  in  the  program 
areas  we  selected.  We  do  this  by  examining  the  lessons  learned  from  experiences  with  past 
weapon  programs.  The  emphasis  is  on  tactical  missiles,  because  that  is  the  area  of  greatest 
cost  and  schedule  variation.  We  were  able  to  analyze  several  of  the  tactical  missile 
programs,  and  we  discuss  which  factors  were  most  important  in  determining  their 
outcomes.  In  the  final  section  of  this  chapter,  we  explore  reasons  for  variations  in  cost  and 
schedule  growth  among  the  tactical  aircraft. 

B .  LESSONS  LEARNED  FROM  MISSILE  PROGRAMS 

This  section  highlights  the  reasons  for  cost  and  schedule  growth  in  selected  tactical 
missile  programs  by  presenting  case  studies  of  various  programs  included  in  our  study. 
The  systems  discussed  here  were  chosen  on  the  basis  of  information  availability  and  do  not 
represent  a  scientific  sample.  They  do,  however,  include  instances  of  programs  with  very 
high  and  very  low  cost  and  schedule  growth.  Among  the  sources  used  in  compiling  this 
information  were  References  [4]  and  [9]. 

At  the  beginning  of  each  program  discussion,  a  chart  similar  to  Figure  IIM  is 
presented  to  indicate  the  position  of  the  program’s  outcome  measures  relative  to  the  rest  of 
the  tactical  missiles.  The  solid  lines  span  the  ranges  of  the  outcome  measures,  from 
minimum  to  maximum.  The  tick  marks  on  the  lines  represent  the  median  of  the  outcome 
measures,  and  the  black  dots  show  the  outcome  measures  for  the  particular  system  being 
depicted.  Such  a  chart  allows  us  to  see,  for  example,  that  the  AIM-7F’s  development  cost 
and  schedule  growth  were  well  above  the  medians  for  the  tactical  missiles  in  our  study. 
Figure  111-2  provides  similar  data  for  the  AIM-7M. 

1 .  Sparrow  Missile  Modifications  (AIM>7F  and  AIM«7M) 

The  Sparrow  missile  is  a  relatively  large  medium-range  air-to-air  missile  that  uses 
semi-active  radar  guidance.  The  Sparrow  has  been  successfully  modified  several  times  to 
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counter  a  changing  threat.  These  modifications  to  the  original  AIM~7C  and  D  versions  are 
largely  evolutionary,  making  improvements  in  reliability,  range,  lethality,  and  clutter 
rejection.  The  Sparrow  has  been  used  in  combat. 

Sparrow  was  a  joint-service  program;  the  Navy  sei-ved  as  the  lead  service  and  the 
Air  Force  supported.  The  Raytheon  Company  has  been  the  prime  contractor  for  all 
versions,  while  General  Dynamics  Corporation  has  been  the  second  source  for  the  AIM-7F 
and  the  A1M-7M. 

Planners  in  the  early  stages  of  the  modifications  considered  them  to  be  technically 
challenging,  but  well-understood.  However,  in  both  cases  (as  well  as  in  the  case  of  the 
earlier  AIM-7E),  the  modifications  involved  the  guidance  and  control  section,  the  most 
technologically  complex  section  of  the  missile.  The  AIM-7F  had  very  high  development 
cost  growth  of  326  percent.  The  schedule  slipped  almost  seven  years,  and  the  system  took 
almost  three  times  as  long  as  originally  planned  from  Milestone  II  to  IOC.  If  modification 
programs  are  complex,  they  can  be  as  difficult  to  develop  as  all-new  systems.  However, 
the  later  modification,  the  AIM-7M,  was  relatively  simple,  and  development  cost  growth 
was  low. 

The  AIM-7F  modification  incorporated  a  switch  to  a  pulse-Doppler  seeker  and  a 
new  government-furnished  fuze.  The  differences  between  the  AIM-7E  and  the  AIM-7F 
included  a  move  from  vacuum  tube  circuits  in  the  guidance  section  to  solid-state  circuits,  a 
larger  rocket  motor,  and  a  larger  warhead.  The  technology  risk  was  underestimated.  The 
technical  problems  necessitated  several  redesign  efforts,  delays  in  flight  testing,  and  more 
launches  of  test  missiles.  Technical  difficulties  were  apparent  during  flight  tests.  After 
seven  launches,  a  major  redesign  effort  was  initiated,  which  resulted  in  a  13-month  gap 
between  contractor  demonstration  test  and  Navy  technical  evaluation. 

The  technical  evaluation  (TECHEVAL)  and  operational  evaluation  (OPEVAL)  for 
the  AIM-7F  was  extended  because  of  the  technical  problems  of  incorporating  the  seeker 
into  the  missile.  A  second  OPEVAL  involving  25  missile  launches  was  conducted.  An 
unusually  large  number  of  pre-production  missiles  were  built  and  tested,  and  the  test 
program  consisted  of  99  launches.  IOC  was  seven  years  later  than  planned. 

The  Navy  decided  on  dual-source  production  of  the  AIM-7F  in  early  1971,  toward 
the  end  of  engineering  development.  The  Navy  was  concerned  about  the  perfonnance  of 
the  system.  Annual  competitions  were  held  from  1977  through  1980.  Raytheon  won  the 
larger  shares  of  the  buys  three  out  of  four  times. 
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The  AIM-7M  program  included  a  competitive  prototype  phase  to  select  between  the 
Raytheon  and  the  General  Dynamics  seeker  designs.  Raytheon  was  selected.  The  AIM-7M 
program  had  a  high  launch  rate  relative  to  other  air-to-air  programs.  The  lack  of  technical 
problems  in  the  AIM-7M  resulted  in  its  relatively  easy  development  schedule  [9], 

Competition  did  not  seem  to  have  much  effect  on  the  costs  for  AIM-7F  and 
AIM-7M,  even  though  both  had  competitive  sourcing  of  the  missile  and  major  subsystems. 
The  monopuise  seeker  in  the  AIM-7M  was  developed  by  Raytheon,  in  competition  with 
General  Dynamics.  In  the  AIM-7F,  any  savings  from  competition  were  overwhelmed  by 
the  technical  difficulties.  The  AIM-7F  program  acquired  more  quantity  than  planned  over  a 
longer  time  than  planned.  Thus,  it  is  not  clear  whether  the  favorable  production  cost  growth 
outcome  was  due  to  the  competition  or  to  program  stability.  The  government  was, 
however,  pleased  with  the  non-cost  aspects  of  the  competition,  including  better  government 
control  and  more  efficient  production  methods. 

The  low  development  cost  growth  of  the  AIM-7M  was  probably  due  to  the  more 
modest  technical  goals  rather  than  to  competition.  However,  the  program  had  high 
production  cost  growth,  despite  only  modest  rate  fluctuations.  The  AIM-7M  had  higher 
total  program  cost  growth  than  the  AIM-7F. 

The  political  environment  does  not  appear  to  have  been  a  major  factor  in  any  of  the 
modifications.  There  were  some  minor  cost  and  schedule  problems  due  to  lack  of  funding 
and  reduction  of  quantities  in  the  AIM-7E  and  AIM-7F  programs.  More  recently,  the 
services  have  withdrawn  funds  from  the  AIM-7M  to  support  the  newer  Advanced  Medium- 
Range  Air-to-Air  Missile  (AMRAAM),  and  this  may  have  caused  problems  with  production 
in  the  AIM-7M. 

2.  Sidewinder  Missile  Modifications  (AIM-9L  and  AIM-9M) 

Both  the  Sidewinder  AIM-9L  and  AIM-9M  modification  programs  encountered 
major  problems  in  development,  as  evidenced  by  large  development  cost  and  schedule 
growth.  Sidewinders  are  short-range  air-to-air  missiles,  the  later  versions  of  which  used 
infrared  guidance. 

Development  of  the  original  Sidewinder  began  in  1950,  and  the  missile  became 
operational  in  1956  (designated  as  the  AIM-9B)  for  the  Navy  and  the  Air  Force.  Over 
60,000  AIM-9B  missiles  were  produced  by  Philco  Corporation  (now  Ford  Aerospace 
Corporation)  and  General  Electric  Company. 
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Several  other  modifications  took  place  before  the  development  of  the  AIM-9L  and 
AIM-9M: 

•  The  Navy  and  the  Air  Force  pursued  separate  modifications  of  the  AIM-9B. 
The  Navy  AIM-9C  was  a  semiactive  radar  homing  variant  that  was  never  fully 
developed.  It  was  eventually  withdrawn  in  favor  of  the  AIM-9D,  which  had  an 
improved  seeker,  an  improved  motor,  and  a  continuous-rod  warhead. 

•  Asa  result  of  the  Navy’ s  changes  to  the  AIM-9B ,  the  Air  Force  hired  Philco  to 
change  the  reference  coils  back  to  previous  values  and  to  make  other 
improvements.  The  modified  missile,  designated  the  AIM-9E,  failed 
acceptance  tests.  The  contractor  offered  a  bargain  price,  and  the  Air  Force 
bought  the  AIM-9Es  and  warehoused  them.  Moreover,  a  factory  was  set  up  in 
Korea  to  convert  the  rest  of  the  AIM-9Bs  into  AlM-9Es  for  the  Korean  armed 
forces,  despite  the  missiles’  test  failures.  Members  of  the  Israeli  armed  forces 
eventually  used  the  missiles,  but  the  Israelis  may  have  made  some  of  their  own 
modifications. 

•  The  Army  modified  the  AIM-9D  into  a  ground-launched  version  called  the 
MIM-72  Chaparral. 

•  The  Navy  added  Sidewinder  Expanded  Acquisition  Capability  to  the  AIM-9D 
to  get  the  AIM-9G,  which  was  produced  by  Raytheon  until  1970. 

•  The  AIM-9H  expanded  on  the  AIM-9G  by  adding  solid-state  circuitry. 

•  The  Air  Force  improved  the  AIM-9E  by  incorporating  solid-state  circuitiy, 
improved  guidance  and  control  units,  and  a  proximity  fuze.  The  new  version 
was  designated  AIM-9J. 

•  The  German  Air  Force  incorporated  its  Viper  missile  technology  into 
Sidewinder.  As  a  result,  Germany  agreed  to  use  the  AIM-9L  as  its  primary  air 
combat  missile  and  to  end  the  Viper  development  program. 

The  AIM-9L  began  in  1970  when  Deputy  Secretary  of  Defense  David  Packard 
directed  the  Navy  and  the  Air  Force  to  join  forces  in  development,  according  to  a 
preliminary  set  of  design  guidelines  outlined  by  IDA.  However,  the  developer  (China 
Lake)  ran  out  of  money  before  the  specified  design  could  be  completed.  Moreover,  there 
were  two  different  versions  of  the  AIM-9L.  The  Navy  version  drew  cryogen  from  a 
launcher-mounted  bottle,  while  the  Air  Force  version  carried  an  internal  cryogen  bottle. 

The  two  modifications  we  examine  here — AIM-9L  and  AIM-9M — have  new 
seekers  capable  of  acquiring  and  locking  onto  a  target  even  in  frontal  aspect  attack.  Earlier 
versions  required  stem  chase  before  lock-on  (4,  Volume  II,  p.  II- 1].  In  both  cases,  the 
fundamental  mistake  made  was  to  view  the  modifications  as  interim  products,  stepping 
stones  to  the  next-generation  system.  This  mistaken  view  produced  an  aggressive 
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acquisition  strategy,  one  that  was  doomed  to  failure.  The  acquisition  strategies  for  the 
AIM-9L  and  AIM-9M  both  included: 

•  few  development  test  articles 

•  use  of  a  single  contractor 

•  substantial  concurrency  (overlap)  between  the  development  test  period  and 
production. 

In  the  case  of  the  AIM-9L,  the  most  important  modifications  were  to  the  seeker 
head.  Initial  testing  failures  made  it  impossible  to  keep  to  the  original  schedule.  In  addition, 
there  were  technical  problems  during  assembly  of  the  first  engineering  unit.  Aimpoint  shift 
was  also  a  severe  problem.  Instead  of  the  33  months  planned,  development  of  the  AIM-9L 
(Milestone  11  to  IOC)  lasted  over  twice  as  long.  Instead  of  a  minimal  30  development 
articles,  192  were  needed.  The  AIM-9L  program  followed  a  fly-before-buy  acquisition 
strategy. 

During  preparation  for  the  Milestone  HI  review  of  the  AIM-9L,  it  was  realized  that 
the  design-to-cost  goal  could  not  be  met,  but  that  unit  costs  for  the  production  missiles 
would  be  at  least  42  percent  higher  than  originally  estimated.  At  this  point,  the  program 
was  drastically  restructured  to  allow  sufficient  time  and  funding  for  strict  adherence  to  the 
“fly-before-buy”  concept.  It  was  decided  to  take  an  exU’a  five  months  to  bring  on  a  second- 
source  contractor  and  to  add  enough  test  missiles  to  make  sure  the  system  worked 
properly.  This  resulted  in  a  389-percent  increase  in  development  costs.  Moreover, 
production  began  two  years  late,  and  IOC  was  four  years  late.  The  technical  problems  and 
the  restructuring  of  the  program  also  resulted  in  high  (107  percent)  production  cost  growth. 

The  AIM-9M  was  later  instituted  to  fulfill  the  original  plan  for  the  AIM-9L,  which 
called  for  countermeasure  rejection.  That  feature  was  eliminated  from  the  AIM-9L  plan 
when  it  was  clear  that  the  program  was  over-running  its  costs.  Moreover,  the  guidance 
section  w'as  repackaged  to  accommodate  a  20-  to  2S-percent  increase  in  complexity  and  to 
improve  producibility.  Dual  sources  for  all  major  assemblies  and  firm-fixed-price 
production  contracts  were  used.  While  development  was  basically  completed  on  time,  there 
was  high  development  cost  growth.  However,  unlike  the  experience  of  the  AIM-9L,  the 
development  problems  of  the  AIM-9M  did  not  spill  over  into  production.  A  difference  of 
the  A1M-9M  production  process  is  that  the  subassemblies  that  make  up  the  missile  were 
shipped  to  Navy  and  Air  Force  facilities  where  the  missile  was  assembled. 

From  the  available  data,  it  is  impossible  to  determine  whether  or  not  funding 
shortages  adversely  affected  the  ability  of  the  programs  to  meet  cost  and  schedule  goals. 


Schedule  delays  in  the  AIM-9L  program  were  clearly  due  to  early  failures  in  testing.  These 
failures  prompted  OSD  to  slow  down  the  program  and  not  allow  it  to  leave  development 
until  problems  were  resolved.  The  results  of  both  the  AIM-9L  and  AIM-9M  programs 
show  the  pitfalls  in  underestimating  the  technical  difficulty  of  modifications  to  guidance  and 
control  systems,  even  when  the  airframes  remain  essentially  unchanged. 

Unlike  the  AIM-9L,  the  AIM-9M  completed  development  essentially  on  time  after 
80  months.  By  February  1976,  when  the  development  plan  for  the  M  version  was 
finalized,  program  officials  knew  that  the  33-month  schedule  for  the  L  was  a  gross 
underestimate,  and  more  time  was  allowed  for  the  M.  The  L  version  was  eventually 
finished  in  82  months,  only  two  months  longer  than  the  .M  version,  but  double  its  original 
estimate. 

A  major  difference  between  the  L  version  and  the  M  version  was  production  rate 
relative  to  plan.  For  the  AIM-9L,  the  restmeturing  of  the  program  meant  that  the  first  9,258 
missiles,  planned  to  take  41  months,  took  94  months  to  complete.  By  contrast,  the 
government  planned  to  produce  7,450  AIM-9M  missiles  in  57  months  and  actually 
produced  them  20  months  sooner.  The  accelerated  production  rate  was  partly  due  to  foreign 
military  sales.  The  technical  problems  in  the  L  version’s  development,  the  unrealistic 
development  schedule  plan,  and  the  slow  production  schedule  combined  to  result  in  a 
system  that  cost  more  than  twice  as  much  as  planned  overall.  The  M  version  had  a  more 
conservative  development  schedule.  It  still  ran  into  development  cost  problems,  although 
not  as  severe  as  those  of  the  L  version.  Overall,  the  accelerated  production  schedule,  as 
well  as  a  vigorous  foreign  military  sales  program,  resulted  in  low  total  cost  growth  for  the 
M  version. 

3.  Phoenix  Missile  (AIM-54A  and  A1M>54C) 

The  Phoenix  missile  has  two  versions — the  AIM-54A,  which  was  a  new  concept 
that  did  not  replace  any  existing  system,  and  the  AIM-54C.  an  improvement  of  the  original 
Phoenix  that  changed  components  from  analog  to  digital. 

The  Navy  devised  a  requirement  in  the  1950s  and  initially  pursued  it  with  the  AAM- 
M-10  Eagle  air-to-air  missile,  which  was  canceled  in  1961.  In  the  1960s,  Hughes  Aircraft 
Company  developed  the  TFX  missile  for  use  on  the  F-1 1  IB  aircraft,  and  it  was  this  missile 
that  became  the  Phoenix  A.  When  the  F-1 1  IB  was  replaced,  the  Phoenix  was  adapted  for 
use  on  the  successor  aircraft,  the  F- 1 4. 
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Figure  ill-5.  Outcome  for  the  Phoenix  AIM-54A 
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Figure  111-6.  Outcome  for  the  Phoenix  AIM-54C 
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The  AIM-54A  began  development  in  1962  and  reached  IOC  in  December  1973. 
IOC  was  later  than  planned,  partly  due  to  problems  with  the  aircraft  platforms — e.g.,  the 
cancellation  of  the  F-1 1  IB  and  schedule  delays  in  the  F-14.  There  was  a  20-month  delay  in 
approval  of  the  missile  for  service,  largely  due  to  delays  in  testing.  The  testing  delays  were 
due  to  unavailability  of  test  platforms  and  late  deliveries  of  the  pilot  production  missiles. 

Development  cost  growth  for  the  AIM-54A  was  54  percent,  and  the  number  of  test 
articles  was  cut  in  order  to  contain  development  costs.  However,  production  cost  growth 
for  the  Phoenix  A  was  only  30  percent,  less  than  the  average  for  new  tactical  missile 
programs.  Reasons  cited  for  cost  growth  in  production  include  underestimation  of  the 
technology  risk,  delays  in  the  test  schedule,  and  engineering  changes  [4].  It  is  also  notable 
that  Hughes  never  faced  any  competition  from  the  advanced  development  phase  on. 

The  AIM-54C  experienced  substantial  problems  in  both  development  and 
production.  The  development  of  the  AIM-54C  improvements  began  in  February  1976,  and 
Milestone  II  was  reached  October  1976.  There  was  considerable  concurrency  between 
development  and  production.  The  build-up  to  fiill-rate  production  was  slow:  the  first  three 
production  lots  contained  fewer  than  100  missiles.  IOC  was  in  December  1986,  more  than 
three  years  later  than  planned,  representing  a  45-percent  development  schedule  slip. 

Development  costs  were  67  percent  higher  than  planned,  and  50  percent  more 
development  articles  were  required.  Reasons  for  the  development  problems  include: 

•  engineering  changes  to  the  guidance  and  control  section, 

•  electronic  counter-countermeasures  (ECCM)  engineering  changes,  and 

•  rework  of  the  igniter  safety  mechanisms  [4]. 

Quality  problems  that  occurred  from  June  through  November  1984  prompted  the 
Navy  to  stop  accepting  deliveries  of  the  missiles  until  the  problems  were  identified  and 
resolved.  In  1984,  the  Navy  began  an  effort  to  develop  a  second  source  for  the  Phoenix  C. 
Raytheon  Company  was  selected  as  the  second  source,  and  head-to-head  competition 
between  Hughes  and  Raytheon  began  in  FY  1989.  However,  production  ended  with  the 
FY  1990  buy. 

The  high  production  cost  growth  for  the  A1M-54C  is  largely  due  to  the  development 
problems  spilling  over  into  production.  IOC  did  not  occur  until  the  eighth  year  of 
production.  In  addition,  it  may  be  that  producibility  issues  were  not  adequately  addressed 
during  development.  In  the  late  1980s,  the  program  was  in  danger  of  being  canceled  so  that 
resources  could  be  devoted  to  the  development  of  the  next-generation  system,  the 
Advanced  Air-to-Air  Missile  (AAAM).  Moreover,  five  years  were  devoted  to  qualifying  a 
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second  source,  for  only  two  years  of  head-to-head  competition.  Even  though  the  system 
was  produced  rapidly,  the  combination  of  the  up-front  cost  of  dual-sourcing  and 
concurrency  increased  the  total  cost  of  the  AIM-54C. 

4.  Advanced  Medium-Range  Air-to-Air  Missile  (AMRAAM) 

The  AMRAAM,  a  replacement  for  the  Sparrow,  provides  double  the  capacity  at  half 
the  size.  It  was  designed  to  provide  all-weather  capability  for  the  F- 16  fighter  and  also  to  be 
used  on  the  F~14,  F-15,  and  F-18  aircraft.  It  is  a  joint-service  program  of  the  Air  Force  (the 
lead  service)  and  the  Navy.  The  technical  difficulty  of  the  program  can  be  considered  to  be 
roughly  comparable  to  the  High-Speed  Anti-Reduction  Missile  (HARM),  but  more  difficult 
than  the  Phoenix  [4].  However,  developers  initially  believed  it  would  be  only  a  little  more 
costly  than  the  F  version  of  the  Sparrow. 


Devtiopment  Production  Total 

Figure  111-7.  Outcome  for  the  AMRAAM 

The  program  began  in  1977  with  concept  definition  and  five  contracts  for 
competitive  advanced  development.  Milestone  I  was  in  November  1978,  and 
demonstration/validation  contracts  were  awarded  to  Hughes  Aircraft  Company  and 
Raytheon  Company  in  February  1979.  Five  Raytheon  and  three  Hughes  missiles  were 
launched.  Hughes  was  selected  as  the  prime  contractor  in  December  1981.  The  Hughes 
FSD  contract  was  fixed-price,  with  priced  options  for  production  lots  1  and  2  and  an 


in- 11 


unpriced  option  for  lot  3.  Milestone  II  was  September  1982 — at  that  time,  Raytheon  was 
selected  as  the  second  source. 

There  was  pressure  to  get  AMRAAM  fielded  quickly  to  synchronize  its  arrival  with 
the  first  F-16s  sent  to  Europe.  The  program  thus  inevitably  had  a  high  degree  of 
concurrency.  Important  tasks  to  demonstrate  the  performance  of  the  missile  were  delayed 
until  full-scale  development.  FSD  was  planned  for  a  period  of  50  months.  Long-lead 
release  occurred  after  only  19  test  launches. 

By  December  1983,  it  was  clear  that  Hughes  could  not  deliver  the  FSD  articles  on 
time.  In  addition,  a  General  Accounting  Office  (GAO)  report  cited  problems  of  cost  growth 
due  to  an  unrealistic  validation  phase  schedule  and  an  overly  ambitious  FSD  schedule. 
Technical  problems  that  contributed  to  the  cost  and  schedule  problems  in  development 
included  problems  in  developing  the  guidance  and  control,  problems  in  missile  integration 
and  assembly,  and  delays  in  flight  testing  due  to  the  delivery  problems.  A  management 
problem  that  contributed  was  pressure  to  make  the  AMRAAM  appear  cheaper  than  its 
competitor,  the  AIM-7M  Sparrow. 

The  first  launch  of  the  AMRAAM  occurred  in  December  1984.  Deliveries  of 
sufficient  missiles  to  fully  equip  the  first  operational  squadron  were  completed  in  early 
December  1990.  Air  Force  IOC  was  achieved  in  September  1991,  62  months  later  than 
planned.  Six  months  of  the  Air  Force  IOC  delay  was  attributed  to  the  lack  of  availability  of 
a  fully  operational  F-15  radar  computer  tape.  Navy  OPEVAL  slipped  from  March  1992  to 
December  1992  because  of  problems  with  the  F/A-18  radar  tape,  then  slipped  to  August 
1993  because  of  issues  identified  during  the  certification  process  of  the  radar  tape.  Phase  I 
of  the  Air  Force  follow-on  test  and  evaluation  (FOT&E)  slipped  from  March  1992  to 
February  1993  because  of  availability  of  test  assets. 

In  the  first  head-to-head  competition,  Hughes  Aircraft  Company  won  59  percent  of 
the  FY  1989  buy.  In  FY  1990,  the  total  quantity  was  divided  evenly  between  Hughes  and 
Raytheon.  In  FY  1991/1992,  Hughes  won  a  981-missilc  buy,  while  Raytheon  received  an 
order  for  810.  In  FY  1993,  Hughes  again  won  a  larger  share,  58  percent  of  the  total. 
However,  in  FY  1994,  Raytheon  was  awarded  the  larger  share.  60  percent  of  the  buy. 

AMRAAM  was  deployed  in  Operation  Desert  Storm  in  February  1991.  Although  it 
was  not  fired,  significant  operational  experience  was  gained,  and  the  missile  greatly 
exceeded  the  mature  MTBMA  requirement.  The  DAB  Milestone  IIIB  review  was  held  in 
May  1991,  and  the  program  was  authorized  to  continue  with  low-rate  production  through 
FY  1992.  The  program  received  approval  for  full-rate  production  in  April  1992.  In 


December  1992,  an  Iraqi  Air  Force  MIG  fighter  was  downed  under  actual  combat 
conditions  using  an  AMRAAM.  A  second  Iraqi  MIG  was  shot  down  in  January  1993. 

AMRAAM  is  a  prime  example  of  the  tendency  to  oversell  programs.  According  to  a 
recent  case  study,  “The  most  important  theme  to  draw  from  this  case  is  the  importance  of 
managing  expectations.  The  chief  cause  of  AMRAAM’s  woes  is  that  managers  vastly 
oversold  the  program  in  tenns  of  cost  and  schedule”  [10]. 

Substantial  production  cost  growth  (59  percent)  on  AMRAAM  went  along  with  the 
development  cost  growth.  Underestimated  technical  difficulty,  combined  with  less 
demonstration/validation  testing  than  originally  planned,  produced  problems  in  production 
as  well.  Concurrency  was  problematic — the  AMRAAM  had  almost  simultaneous 
development  test  and  evaluation  (DT&E)  and  initial  operational  test  and  evaluation 
(lOT&E).  An  unrealistic  FSD  flight  test  schedule  further  compounded  the  problems. 

In  1986,  Congress  capped  both  the  FSD  and  production  programs.  FSD  was 
completed  within  the  cost  cap,  but  Congress  had  to  increase  the  production  cost  cap  twice. 
Low-rate  initial  production  began  in  June  1987,  more  than  three  years  later  than  planned. 
The  DAB  reviewed  cost  and  schedule  breaches  in  December  1989.  A  drastic  cutback  in 
production  quantity  and  a  major  schedule  stretchout  resulted  in  a  production  schedule 
stretch  of  173  percent  for  this  program.  This,  combined  with  dual  sourcing  at  less  than  full 
rate,  resulted  in  high  production  cost  growth. 

The  combination  of  reduced  production  quantities  and  expanded  schedules,  both 
exhibited  in  the  AMRAAM  program,  is  a  familiar  recipe  for  cost  growth;  however,  the  jury 
is  still  out  on  the  program.  Approval  for  full-rate  production  was  not  given  until  April 
1992.  Future  buys  may  be  less  costly  and  the  final  cost  growth  figure,  lower. 

5.  Maverick  Missile  (AGM-65A  and  AGM.65D/G) 

The  Maverick  is  an  air-to-ground  missile  with  a  close  air  support  mission.  Two 
versions  of  the  missile  were  used  for  our  study,  AGM-65A  and  AGM-65D/G. 

The  early  version  of  the  Maverick  used  the  total  package  procurement  (TPP) 
concept.  The  TPP  negotiations  began  in  the  late  1960s,  a  time  when  other  TPP  programs 
were  having  major  problems.  The  contract  definition  phase  was  extended  about  a  year  to 
allow  intensive  negotiations  toward  the  goal  of  designing  “the  perfect  contract.”  The 
original  FSD  contract  contained  options  for  three  production  buys. 
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The  contract  included  incentives  for  good  performance  and  penalties  for 
unacceptable  performance.  For  example,  a  missile  hit  rate  of  90  percent  resulted  in  a  $3 
million  incentive  fee.  If  the  hit  rate  was  80  percent,  no  bonus  was  given,  and  if  the  hit  rate 
was  70  percent,  a  $1.5  million  penalty  was  applied.  If  the  hit  rate  got  below  70  percent,  ihe 
contractor  was  required  to  provide  free  replacement  hardware  for  additional  testing. 
Because  the  1970s  were  a  time  of  high  inflation,  the  contract  included  an  escalation 
clause  [3]. 

From  the  contractor’s  point  of  view,  one  of  the  great  advantages  of  TPP  was 
stability.  Once  the  requirements  for  test  success  and  cost  were  defined,  the  contractor  could 
meet,  them  with  a  minimum  of  interference,  and  without  constant  contract  negotiations  and 
reviews.  The  contractor  also  believed  that  the  acceptance  and  execution  of  Value 
Engineering  proposals  saved  money.  Value  Engineering  was  a  program  designed  to  allow 
contractors  to  share  in  cost  savings  from  efficient  operation. 

However,  despite  this  use  of  TPP  (which  has  proven  completely  unsuccessful  in 
other  applications),  there  was  very  little  concurrency  in  the  program.  The  first  production 
option  was  exercised  in  July  1971,  one  month  before  testing  was  complete.  Testing  was 
particularly  vigorous,  with  a  launch  rate  of  5.2  missiles  per  month  during  Category  II 
testing.  Perhaps  this  lack  of  concuirency  is  why  the  program  did  reasonably  well,  despite 
TPP.  In  production,  both  cost  and  schedule  proceeded  essentially  according  to  plan. 

The  B  version  of  the  system  had  better  optics,  but  it  still  used  daylight  television 
guidance.  While  the  A  and  B  versions  had  problems  in  daylight  if  used  improperly,  they 
were  never  intended  for  night  use.  The  second  Maverick  in  our  study  is  the  Imaging 
Infrared  (HR)  Maverick,  which  uses  infrared  guidance  and  is  designed  for  day/night,  all- 
weather  operation. 

The  HR  Maverick  is  designated  AGM-65D/G  in  the  SAR,  but  there  are  other 
versions  as  well: 

•  AGM-65D,  the  first  infrared  Maverick; 

•  AGM-65E,  a  Navy  and  Marine  Corps  version  with  semiactive  laser  seeker; 

•  AGM-65F,  a  Navy  anti-ship  version;  and 

•  AGM-65G,  a  modification  of  the  D  version  with  a  bigger  warhead. 

The  IIR  Maverick  had  a  prototype  program  that  took  18  months  from  the  start  to  the 
first  launch  of  a  prototype  missile.  There  was  no  competitive  fly-off.  A  total  of  790  flight- 
hours  of  captive  testing  was  completed  before  the  award  of  the  FSD  contract.  The 
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unusually  long  period  of  time  between  the  prototype  launch  program  and  the  start  of  FSD 
was  devoted  to  an  extended  captive-flight  program  that  included  790  flight  hours  of  tests  in 
various  operational  environments  and  producibility  and  cost  studies. 

The  OR  Maverick  had  its  Milestone  II  review  in  September  1976,  but  engineering 
development  did  not  start  until  October  1978  due  to  an  OSD-directed  funding  cut.  (As  we 
will  see,  this  helps  to  explain  the  high  development  schedule  growth,  but  low  development 
cost  growth.)  The  original  design  intent  was  to  use  forward-looking  infrared  (FLIR) 
equipment  to  acquire  the  target  and  a  correlator  to  bring  the  Maverick  seeker  onto  the  target, 
llie  program  manager,  fearing  that  the  cost  of  equipment  would  kill  the  program,  asserted 
that  no  electronics  other  than  a  small  TV  display  were  required  to  use  HR  Maverick.  Early 
tests  were  run  without  other  equipment,  arid  serious  problems  were  encountered  with  target 
location  aiid  target  lock-on. 

After  its  Milestone  IIIA  review  in  March  1982,  the  HR  Maverick  received  a  partial 
release  for  pilot  production.  In  September  1982,  the  HR  Maverick  received  full  go-ahead 
for  pilot  production.  There  were  problems  with  operational  suitability,  although  not  with 
operational  effectiveness.  The  Hughes  production  line  shut  down  from  March  1984  to 
December  1984  to  correct  quality  problems.  Full-rate  production  was  not  authorized  until 
March  1986.  IOC  was  achieved  a  month  earlier,  in  Febmary  1986. 

Meanwhile,  a  second-source  request  for  proposals  (RFP)  was  released  in  1982, 
and  Raytheon  was  awarded  a  second-source  contract  in  May  1983.  Modifications  and  add¬ 
ons  to  the  contract  raised  the  price  to  $76.2  million  for  15  missiles.  Hughes  produced  sole 
source  from  FY  1982-1986.  Hughes  won  65.7  percent  of  the  buy  during  the  first  year  of 
competition  (FY  1987);  Raytheon  won  63.7  percent  of  the  buy  in  FY  1988;  and  Hughes 
won  59  percent  of  the  buy  in  FY  1988.  In  early  1989,  the  government  decided  to  make 
future  awards  on  a  winner- take-all  basis.  Through  the  split  buys,  the  government  received 
considerable  data  on  contractor  costs,  and  with  lower  planned  quantities,  it  could  not  afford 
to  support  two  contractors.  Hughes  won  the  entire  FY  1990  procurement,  and  Raytheon 
won  the  contract  to  replace  5,255  missiles  used  in  Desert  Storm.  These  were  the  last 
missiles  bought. 

The  program  had  high  development  schedule  growth.  It  took  almost  twice  as  long 
to  develop  the  HR  Maverick  than  planned.  The  time  between  Milestone  II  and  Milestone 
IIIB  was  72  months,  rather  than  the  planned  42,  mostly  due  to  a  funding  cut  that  delayed 
the  actual  start  of  engineering  development.  This  delay  may  have  actually  contributed  to  the 
technical  success  of  the  program.  It  is  likely  that  the  Air  Force’s  demands  may  have 
outstripped  the  technical  capability  of  the  time.  Unusually  warm  weather  in  winter  1981 
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delayed  winter  site  tests  for  one  year.  The  time  between  Milestone  niB  and  ICX!  was  41 
months,  not  the  IS  months  planned.  This  was  largely  due  to  underestimation  of  the 
technical  difficulty  of  the  program. 

The  HR  Maverick  program  exhibited  low  development  cost  growth,  and  its 
production  cost  growth  was  below  the  median.  The  number  of  development  test  articles 
was  reduced  to  save  money.  If  they  had  been  increased,  some  of  the  producibility  problems 
may  have  been  resolved  earlier.  Pilot  production  continued  for  a  considerable  time  before 
IOC  was  achieved,  so  there  was  a  lot  of  overlap  between  development  and  production.  A 
1982  GAO  report  following  up  on  the  unit  cost  report  breach  stated  that  the  largest  single 
cause  of  unit  cost  growth  was  an  increased  estimate  of  guidance  unit  cost  reflecting 
technical  and  cost  problems  in  the  research  and  development  (R&D)  phase.  The  shutdown 
of  the  Hughes  production  line  to  resolve  quality  problems  also  may  have  contributed  to 
production  cost  growth.  The  quantity  planned  at  Milestone  n  was  never  achieved,  and  there 
v/as  93  percent  schedule  growth. 

The  results  of  the  dual-sourcing  strategy  are  uncertain.  There  does  appear  to  be 
some  decrease  in  unit  cost,  but  it  was  not  clear  whether  or  not  the  up-front  investment 
would  be  repaid. 

Both  versions  of  the  Maverick  had  very  low  cost  growth  as  compared  with  other 
tactical  missiles.  The  Maverick  AGM-65A’s  low  cost  growth  is  especially  remarkable  in 
light  of  the  fact  that  it  used  a  total  package  procurement  strategy.  Other  programs  using  this 
strategy  have  had  very  high  cost  growth.  The  program  used  an  incentive  contract  with  an 
inflation  escalator  clause  to  cushion  the  effect  of  the  very  high  inflation  of  the  time  In 
addition,  the  AGM-65A  was  subjected  to  considerable  testing,  and  production  was  not 
permitted  to  proceed  until  tests  were  essentially  complete.  In  production,  the  program 
adhered  to  its  production  schedule  and  procured  slightly  more  quantity  than  planned. 

The  HR  Maverick  was  prototyped  and  tested  before  FSD,  so  more  accurate  cost 
estimates  could  be  made  at  Milestone  H.  There  was  a  two-year  hold  in  engineering 
development  due  to  a  funding  cut.  This  explains  the  anomaly  of  high  development  schedule 
growth  but  low  development  cost  growth.  Despite  a  production  rate  lower  than  planned 
(high  PSS),  production  costs  were  probably  contained  through  dual  sourcing  with  a  buyout 
at  the  end  of  the  program. 
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6.  Harpoon  Missile  (AGM-84) 


The  Harpoon  is  an  interesting  case  study,  because  it  was  a  crash  development 
program.  The  Navy  pushed  the  program  schedule  ahead  by  three  years  to  counter  an 
immediate  threat  from  ships  equipped  with  anti-ship  missiles.  The  result  was  that  the 
development  program  was  completed  faster  than  other  similar  systems  and  within  its 
development  cost  estimates.  However,  cost  growth  in  production  has  been  high. 


Development  Production  Total 

Figure  11-10.  Outcome  for  the  Harpoon  AGM-84 

The  Harpoon  began  as  an  all-weather,  air-to-surface,  anti-ship  missile.  The 
requirement  was  expanded  in  1967  to  include  a  ship-launched  variant  (RGM-84)  and  again 
in  1972  to  include  a  submarine-launched  version  (UGM-84A).  The  Harpoon  has  proven  to 
be  excellent  in  performance  and  operations. 

McDonnell  Douglas  Corporation  had  experience  in  developing  concepts  for  anti¬ 
ship  missiles.  In  March  1970,  concept  development  began,  and  in  June  1971,  a  design 
contract  was  awarded  after  competitive  bids.  The  first  flight  was  in  February  1972. 
Milestone  IIA  was  held  in  May  1972,  and  the  FSD  contract  was  awarded  in  June  1973. 
However,  there  was  no  official  development  estimate  until  a  year  later,  in  the  June  1974 
SAR.  This  development  estimate  of  production  costs  was  considerably  lower  than 
estimates  shown  in  the  SARs  both  immediately  before  and  immediately  after  June  1974 
(see  Figure  III- 11). 


PAUC  Procurement 

1970$  Quantity 


Sep  71  Deo  71  Dec  72  Dec  73  Jun  74  Dec  74  Dec  75  Dec  76  Dec  77 

SAR  Date 

Figure  111-11.  Estimates  of  Procurement  Average  Unit  Cost  (PAUC)  and 
Procurement  Quantity  for  the  Harpoon 

The  operational  evaluation  process  (from  June  1 975  to  March  1977)  uncovered 
quality  problems,  which  were  eventually  resolved.  As  a  safeguard  against  problems  due  to 
the  rapid  nature  of  development,  a  comprehensive  cost  reduction  program  was  activated, 
and  Value  Engineering  concepts  were  funded.  The  first  IOC  occurred  in  June  1977,  for 
FF-1052  class  ships.  There  was  a  high  degree  of  concurrency  in  the  program — initial 
production  go-ahead  occurred  7  months  before  the  beginning  of  Navy  technical  evaluation. 
Long-lead  release  occurred  before  the  start  of  OPEVAL. 

Development  thus  was  considerably  faster  than  typical;  the  missile  was  flying  less 
than  two  years  after  the  beginning  of  concept  development,  and  IOC  was  just  over  five 
years  past  Milestone  IIA.  How  was  this  achieved?  Perhaps  the  program  benefitted  from 
McDonnell  Douglas’s  past  work  in  anti-ship  missile  concepts.  In  addition,  [4]  cites  the  fact 
that  the  sustainer  engine  was  made  a  separate  program,  and  that  there  was  an  additional 
Milestone,  HB,  for  pilot-line  production  go-ahead. 

Development  schedule  growth  was  about  average  for  the  group  of  systems,  but  the 
total  schedule  of  50  months  is  still  much  faster  than  for  a  typical  new  tactical  munitions. 
Development  cost  growth  was  low,  perhaps  because  of  the  timing  of  the  development 
estimate.  The  large  (85  percent)  production  cost  growth  could  be  due  to  development 
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problems  spilling  over  into  production  and  to  production  schedule  stretchout  (it  took  66 
percent  longer  than  planned  to  produce  the  development  estimate  quantity). 

The  experience  of  the  Harpoon  suggests  that  development  programs  can  be  pushed 
in  critical  times,  particularly  when  a  single  experienced  contractor  is  selected  and  proven 
technology  is  used.  However,  there  is  a  price  to  be  paid  for  development  haste,  and  this 
price  often  must  be  paid  in  production.  The  crash  development  program  contributed  to  the 
difficulty  in  estimating  costs  of  manufacturing  complex  subassemblies,  the  underestimation 
of  rate  tooling,  the  volume  of  engineering  change  orders,  and  high  government  in-house 
costs.  More  prototyping  probably  would  have  helped  to  avoid  some  of  the  difficulties  in 
producibility  of  the  seeker  and  in  assembly  integration. 

7 .  High-Speed  Anti-Radiation  Missile  (HARM) 

The  HARM  is  an  air-to-ground  missile  that  was  developed  jointly  by  the  Navy  (lead 
service)  and  the  Air  Force.  HARM  was  developed  at  the  Naval  Weapons  Center,  China 
Lake,  California.  It  evolved  from  the  Shrike  (AGM-45)  and  the  early  Anti-Radiation 
Missiles  (AGM-TS). 


Figure  111-12.  Outcome  for  the  HARM 

Milestone  I  for  HARM  was  in  October  1972,  and  Texas  Instruments,  Incorporated, 
was  awarded  the  contract  for  integration  in  May  1974.  The  first  HARM  prototype  missile, 
an  aerodynamic  flight  test  vehicle,  was  launched  in  January  1976.  The  first  prototype 
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guided  missile  was  launched  in  October  1976.  A  total  of  29  advanced  development  missiles 
were  tested.  Out  of  18  prototype  firings  during  engineering  development,  13  were  complete 
successes.  Corrective  actions  were  taken  before  procurement  funds  were  committed.  At  the 
Milestone  II  meeting  (January  1977),  the  HARM  program  was  retained  in  advanced 
development  for:  (1)  prototyping  expanded  capability  in  frequency  and  aerodynamic 
maneuverability  and  (2)  conducting  a  cost-effectiveness  analysis  of  HARM  against 
available  alternatives.  Just  over  a  year  later  (Febraary  1978),  the  program  received  approval 
to  move  into  full-scale  engineering  development.  The  first  launch  was  in  April  1979,  and 
low-rate  initial  production  began  in  late  1981. 

Three  other  contractors — Ford  Aerospace  Corporation,  Raytheon  Company,  and 
Bendix  Corporation — took  the  Texas  Instruments  hardware  and  design  and  developed 
proposals  for  manufacturing  the  system.  In  December  1982,  the  Navy  threatened  to  cancel 
the  program  unless  Texas  Instruments  substantially  reduced  program  costs.  Texas 
Instruments  agreed  to  several  producibility  engineering  proposals  that  reduced  production 
costs  by  around  3  percent  and  also  agreed  to  provide  firm  price  quotations  for  two  years 
into  the  future.  The  Navy  proposed  a  dual-source  plan  at  Milestone  HI,  but  OSD  rejected 
the  plan  and  called  instead  for  vendor-level  competition,  with  the  prime  contractor 
responsible  for  cost  control.  Competition  was  never  implemented,  but  Texas  Instruments 
dropped  its  price  substantially,  from  $937.5  thousand  per  unit  in  1981  to  $514.4  thousand 
per  unit  in  1982  and  $313.8  thousand  per  unit  in  1983.  IOC  was  achieved  in  November 
1983.  A  total  of  40  HARMs  were  launched  in  combat  against  Libya  in  March  and  April 
1986. 

Congress  rejected  a  multiyear  procurement  strategy  for  HARM  on  the  grounds  that 
the  assumed  production  rate  was  more  ambitious  than  was  likely  to  occur,  and  the  potential 
savings  from  multiyear  procurement  were  too  small.  A  design-to-cost  procurement  strategy 
was  used,  but  the  design-to-cost  threshold  was  breached  only  four  years  into  the  program. 

Development  schedule  growth  was  high  compared  to  other  systems  in  this  group, 
as  was  development  cost  growth.  Reasons  for  this  included  the  need  for  increased  funds 
for  A-6E/HARM  integration  and  the  expanded  capability  required.  In  production,  cost 
growth  was  slightly  below  the  median  for  the  class.  Schedule  growth  in  production  was 
slightly  higher  than  the  median.  The  system  exceeded  its  technical  performance 
requirements. 

The  requirement  for  flexible  software,  while  it  added  to  the  capability  of  the  system, 
was  difficult  to  achieve.  Because  of  the  technical  difficulty  of  the  flexible  software 
requirement,  a  four-year  advanced  development  phase  was  planned,  but  even  this  was  not 
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enough.  Prototyping  took  some  extra  time,  and  fixing  problems  identified  added  to 
development  costs,  but  the  problems  were  fixed  early.  Perhaps  as  a  result,  production, 
where  most  of  the  money  is,  stayed  closer  to  plan  than  is  typical  for  tactical  munitions. 
Finally,  even  though  the  dual-sourcing  strategy  was  rejected,  Navy  officials  have  indicated 
that  they  believe  that  the  threat  of  competition  led  to  more  reasonable  prices  from  the  prime 
contractor. 

8.  Hellfire  Missile  (AGM-114) 

The  AGM-1 14  has  several  distinctive  features:  it  is  very  light  for  an  air-to-surface 
missile,  it  has  laser  guidance,  it  is  launched  from  a  helicopter  (the  AH-64),  and  it  is 
developed  by  the  Army.  A  comparatively  large  number  of  test  missiles  relative  to  similar 
programs  were  procured  and  launched. 


Figure  111-13.  Outcome  for  the  Hellfire  AGM-114 

An  interesting  feature  of  the  Hellfire  program  is  the  establishment  of  a  second 
source  at  no  apparent  cost  to  the  government.  Up  until  1988,  according  to  [4],  the  net  effect 
of  the  dual-source  capitalization  strategy  and  the  pressure  of  competition  was  a  wash. 
Prices  were  essentially  the  same  as  if  there  had  been  no  competition. 

The  government  followed  a  split-buy  strategy  in  FY  1984-1989.  In  FY  1990, 
Rockwell  International  Corporation  won  the  total  quantity  for  FY  1990,  with  firm-fixed- 
price  options  for  FY  1991  and  FY  1992.  Rockwell  is  in  a  sole-source  position  for  FY 


in-22 


1991-93.  An  improved  version,  the  Hellfire  Optimized  Missile  System  (HOMS),  will  be 
produced  by  Martin  Marietta  Corporation. 

The  Hellfire  program  began  advanced  development  in  1972  with  competitive 
prototyping.  In  1974,  Rockwell  won  a  development  contract  for  a  tri-service  laser  seeker 
and  entered  FSD  in  1976.  Martin  Marietta  submitted  an  unsolicited  proposal  for  a  low-cost 
alternative  seeker  that  had  been  developed  privately,  and  Martin  Marietta  won  a  competition 
over  Rockwell  in  1977.  The  technology  to  develop  Hellfire  was  essentially  available  at  the 
time  of  development;  the  development  program  did  not  greatly  push  the  state  of  the  art.  The 
development  program  took  much  longer  than  is  typical  for  an  air-to-surface  missile. 
Reasons  cited  in  the  SAR  for  FSD  delays  include  “reduction  of  RDT«&E  funding,  delays  in 
procurement  funding,  and  delays  in  testing  caused  by  late  delivery  of  hardware  and 
correction  of  deficiencies  revealed  in  earlier  tests." 

The  Army  was  delegated  authority  for  Milestone  ID,  and  Hellfire  was  approved  for 
production  in  March  1982.  Initial  production  contracts  were  awarded  in  FY  1982  to 
Rockwell  for  missile  buses  and  to  Martin  Marietta  for  seekers.  Later  that  year,  it  was 
decided  to  have  both  contractors  produce  the  whole  system  in  order  to  have  competition. 
Each  contractor  produced  a  limited  number  of  complete  systems  (“all-up-rounds”)  for 
certification  in  1983,  and  head-to-head  competition  began  in  1984.  In  order  to  keep  the 
production  base  intact  for  both  contractors,  a  split-buy  strategy  was  adopted.  In  the  first 
year  of  competition,  each  contractor  was  guaranteed  at  least  40  percent  of  the  buy.  From 
FY  1985  through  FY  1988,  each  contractor  was  guaranteed  a  25-percent  share  of  the  buy. 
Each  contractor  was  allowed  to  build  in  the  cost  of  establishing  the  production  line  into  the 
missile  buys. 

The  two  prime  contractors  at  first  alternated  in  winning  the  larger  shares  of  the 
buys.  Still,  major  components,  including  the  warhead,  body,  gyros,  and  rocket  motor  are 
still  supplied  by  a  single  vendor.  Martin  Marietta  tried  to  qualify  a  second  source  to 
compete  with  the  rocket  motor  producer.  However,  on  the  threat  of  competition,  the 
original  producer  lowered  its  prices  to  the  point  where  it  was  not  worthwhile  to  have 
competition.  Martin  Marietta  has  been  successful  in  qualifying  a  second  source  for  the 
missile  control  section.  More  recently,  Rockwell  has  won  the  total  quantity  and  is  the  sole 
producer  for  FY  90-93. 

The  HOMS  began  development  in  July  1989  and  is  slated  to  begin  production  in 
May  1993.  We  have  not  included  funding  for  this  program  in  our  estimates.  We  also  have 
not  included  funding  for  Longbow  Hellfire,  which  is  broken  out  for  reporting  purposes  as 
a  separate  program  and  does  not  have  enough  data  to  be  included  in  this  study. 
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The  development  period  (Milestone  11  to  IOC)  was  about  ten  years.  The  schedule 
was  delayed  by  about  two  years  during  FSD,  due  to  funding  reductions  and  delays  in 
testing  caused  by  late  delivery  of  hardware  and  correction  of  deficiencies  revealed  in  earlier 
tests.  Problems  in  production  start-up  delayed  production  validation  testing  for  six  months. 
The  system  satisfies  all  its  mission  requirements  except  missile  weight.  At  99.8  pounds,  it 
is  4.8  pounds  over  its  required  weight,  and  it  was  not  considered  feasible  to  reduce  the 
weight  without  degrading  performance. 

The  program  appears  to  have  been  successful.  Development  cost  growth  was  low, 
only  22  percent.  Production  cost  growth  was  60  percent.  Competition  did  not  obviously 
reduce  costs,  but  it  also  did  not  increase  them  and  probably  contributed  to  the  stability  of 
the  program. 

9.  Tube*launched,  Optically-trackedl,  Wire-guided  (TOW)  and  TOW  2 

Missiles 

TOW  is  a  crew-portable  or  vehicle-mounted  heavy  anti-tank  weapon.  The  basic 
TOW  program  began  advanced  engineering  development  in  April  1963.  Hughes  designed 
the  system  and  has  been  the  principal  producer  of  the  missile  and  launcher.  Initial 
production  contracts  were  awarded  to  Hughes  for  missiles  and  launchers  in  November 
1968  and  to  Chrysler  Corporation  for  missiles  in  January  1969.  IOC,  as  measured  by  first 
unit  equipped,  was  achieved  in  September  1970.  Hughes  won  a  winner-take-all 
competition  against  Chrysler  in  November  1971. 

A  two-step  TOW  improvement  program  was  initiated  in  August  1979  to  meet  the 
threat  of  advanced  enemy  armor.  The  first  step,  improved  TOW,  added  a  larger  warhead 
with  better  performance  against  armor.  Planned  quantities  of  improved  TOW  missiles  were 
noi  procured  before  initiation  of  step  two,  TOW  2.  A  development  contract  for  TOW  2  was 
awarded  to  Hughes  Aircraft  Company  in  December  1978.  The  design  features  a  larger 
warhead  with  a  probe,  thermal  beacon,  and  improved  flight  motor  designed  to  defeat  the 
frontal  armor  of  Soviet  battle  tanks.  A  microprocessor-based  digital  missile  guidance  set 
provides  improved  guidance.  TOW  2  was  approved  for  full-scale  production  in  September 
1981. 

TOW  schedule  delays  during  advanced  development  and  FSD  amounted  to  about 
two  years,  but  it  was  still  possible  to  complete  the  program  (advanced  development  to  IOC) 
in  about  seven  years.  The  main  factors  in  development  delays  were  a  delay  in  completing 
development  of  the  missile  container,  and  low  ^liability  factors. 
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Figure  lll‘‘14.  Outcome  for  the  TOW  Missile 
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Figure  111-15.  Outcome  for  the  TOW  2  Missile 


Total  TOW  production  quantity  for  the  Army  decreased  from  232,614  missiles  to 
137,275.  These  drastically  reduced  procurement  quantities  were  probably  a  key  factor  in 
the  high  rate  of  production  cost  growth.  The  production  schedule  did  not  correspondingly 
decrease,  but,  in  fact,  substantially  increased.  Thus,  production  schedule  stretch  is  high, 
290  percent. 

TOW  followed  a  dual-source  acquisition  strategy,  with  split  buys  between  Hughes 
and  Chrysler  Corporation.  After  a  winner-takc-ull  competition,  multiyear  procurement 
contracts  were  awarded  to  Hughes  in  1971  and  1975. 

The  technical  performance  of  both  TOW  and  TOW  2  is  highly  successful.  The 
TOW  is  within  10  percent  of  the  development  estimate  weight  and  met  all  other  operational 
requirements.  TOW  2  has  met  all  weight,  range,  and  reliability  requirements,  and  accuracy 
has  exceeded  requirements.  Development  cost  growth  of  39  percent  is  due  to  engineering 
changes  for  the  TOW  2A  and  TOW  2B  enhancements,  according  to  the  program  manager. 
There  has  been  no  production  cost  growth. 

The  TOW  program  defined  the  threat  and  operational/technical  requirements  early  in 
the  process,  and  technologies  were  limited  to  those  considered  to  be  proven.  Modifications 
were  implemented  incrementally.  Concept  development  took  place  in  a  competitive 
environment;  procurement  included  dual-sourcing  and  multiyear  procurement.  "Should 
cost”  studies  were  used  to  evaluate  competitive  bids.  Competitive  or  dual-sourcing  of 
launcher  and  night  sight  were  also  employed.  Despite  all  this,  total  program  costs  grew  71 
percent.  Among  the  factors  that  went  wrong  are  program  stretchout,  a  drastic  reduction  in 
quantities  procured,  and  problems  in  development  of  the  thermal  night  sight. 

The  TOW  2  program  was  right  on  schedule.  It  achieved  a  rapid  development 
schedule,  with  some  development  cost  growth  due  to  engineering  changes.  The  production 
line  was  stopped  in  1984  for  non-compliance  with  Military  Standards.  Nonetheless, 
adherence  to  the  planned  production  schedule  and  increased  quantity  contributed  to  low 
production  cost  growth. 

10.  Multiple  Launch  Rocket  System  (MLRS) 

The  MLRS  program  employed  an  unusual  acquisition  strategy  in  several  respects. 
The  development  program  was  multinational,  and  the  schedule  was  concurrent  and 
accelerated.  MLRS  was  successful.  The  development  program  was  completed  on  time  with 
a  minimum  of  cost  growth,  and  a  multiyear  production  contract  has  contained  costs  in 
production.  Total  program  costs  are  actually  10  percent  below  plan. 
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Figure  IIUIS.  Outcome  for  the  MLRS 


MLRS  is  an  Army  system  designed  to  deliver  firepower  quickly  against  critical, 
time-sensitive  targets.  The  Army  began  development  in  the  mid-1970s  and  awarded  five 
concept  definition  contracts  in  March  1976.  A  Milestone  I  review  was  held  in  January 
1977.  The  Secretary  of  Defense  directed  the  Army  to  continue  studying  means  of 
accelerating  the  program  and  to  give  high  priority  to  standardizing  the  weapon  system  for 
the  North  Atlantic  Treaty  Organization  (NATO).  A  combination  of  manpower  shortages 
and  the  need  to  counter  large  numbers  of  Soviet  weapons  made  the  program  an  Army -wide 
priority.  The  program  enjoyed  strong  congressional  support.  In  fact.  Congress  provided  $5 
million  in  FY  1977  for  the  program,  $4  million  more  than  the  Secretary  of  Defense  had 
requested.  The  program  office  resolved  to  set  modest  technical  goals  and  then  exceed  them 
[11].  The  Army’s  plan  for  acceleration  provided  for  advanced  development,  but  full-scale 
engineering  development  (FSED)  only  if  required.  If  development  risks  appeared  tolerable 
after  the  validation  phase,  production  could  begin  without  FSED. 

The  initial  design  process  was  iterative.  The  government  rejected  a  strategy  of 
providing  only  general  specifications  and  concluded  that  the  Missile  Command  had  more 
detailed  technical  knowledge  than  any  single  contractor.  Thus,  the  government  called  for  an 
iterative,  cooperative  design  process. 
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Because  of  the  urgency  of  the  program,  the  program  manager  received  some 
dispensations  from  standard  procedures.  Some  routine  reports  and  meetings  were  omitted, 
and  lawyers  and  contract  specialists  were  assigned  to  the  program  full-time,  rather  than 
having  to  be  drawn  from  a  pool  shared  by  other  contract  specialists. 

In  1977,  two  contractors,  the  Boeing  Company  and  Ling-Tempco-Vought  (LTV) 
Aerospace  and  Defense  Company  were  selected  to  build  prototype  launched  systems  with 
associated  flight  test  equipment  and  hardware.  The  competitive  shoot-off  had  good  results. 
LTV  was  selected  as  the  prime  contractor  for  the  maturation/initial  production  phase  in  May 
1980.  DSARC  III  was  held  in  May  1980;  the  normal  Milestone  II  review  was  omitted 
because  of  the  acceleration  of  the  program.  The  technology  used  was  relatively  simple.  The 
Army  was  willing  to  accept  performance  somewhat  less  than  planned  to  field  a  system 
quickly. 

There  was  a  five-year  contract  awarded  to  LTV  in  September  1983,  following  a 
should-cost  analysis.  The  multiyear  contract,  with  options,  covered  all  self-propelled 
launchers/loaders  and  rockets  for  the  life  of  the  program  as  approved  at  that  time.  The  GAO 
concluded  that  the  multiyear  contract  had  saved  $166.8  million,  due  to  purchasing  certain 
raw  materials  and  components  earlier,  and  in  more  economical  quantities,  than  would  have 
been  done  under  annual  contracts.  A  second  multiyear  contract  was  awarded  in  June  1989 
covering  the  FY  1989-1993  buys.  The  multiyear  contracts  probably  contributed  to  the 
stability  of  the  program. 

The  MLRS  program  exhibited  little  development  schedule  growth  and  schedule 
delay  was  only  three-months  (to  satisfy  a  German  requirement  for  a  scatterable  mine 
warhead).  In  July  1979,  a  memorandum  of  understanding  (MOU)  covering  the  intended 
adoption  of  MLRS  was  signed  by  France,  Germany,  the  United  Kingdom,  and  the  United 
States.  The  contractor  believes  that  the  international  nature  of  the  program  helped  to 
maintain  its  stability.  IOC  slipped  only  4  months  from  the  original  estimate,  but  the 
definition  of  IOC  changed.  The  initial  SAR  defined  IOC  as  eighteen  launchers  with  60 
rockets  per  launcher,  while  the  final  definition  was  nine  launchers  with  60  rockets  per 
launcher. 

The  demonstrated  technical  characteristics  of  system  accuracy  and  nuximum  range 
are  below  the  planning  estimate.  However,  the  December  1980  SAR  reported  that  the 
existing  maximum  range  was  acceptable,  and  that  it  would  not  be  cost-effective  to  make  the 
changes  required  to  increase  it. 
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Both  the  U.S.  and  its  allies  used  MLRS  during  Operation  Desert  Storm.  According 
to  the  1992  SAR,  the  system  “performed  extremely  well... when  significant  numbers  of 
MLRS  launchers  were  deployed.  All  significant  requirements  were  met,  and  in  most  cases 
exceeded  for  readiness,  reliability,  accuracy,  and  maintainability,” 

The  best  estimate  of  cost  growth  for  this  program,  and  the  one  used  in  this  study,  is 
from  Reference  [5].  We  did  not  update  this  estimate,  because  of  the  difficulty  of  deriving 
cost-quantity  relationships.  Standard  data  sources  group  the  prices  of  the  launchers  and  the 
rockets  into  one  dollar  amount.  The  quantity  of  rockets  per  launcher  changed  over  the 
course  of  the  program,  so  it  is  difficult  to  develop  an  appropriate  price-improvement  curve. 
In  the  study  documented  in  [5],  IDA  obtained  detailed  data  on  rockets  and  launchers  and 
developed  separate  price  improvement  curves  based  on  these  data.  The  production  cost 
growth  estimate  based  upon  those  price  improvement  curves  is  used  in  this  study. 

Several  lessons  were  learned  from  the  MLRS  experience: 

1 .  Threat  analysis  led  to  a  climate  for  MLRS  to  be  developed  under  a  priority 
system,  including  concurrency.  While  this  is  often  a  recipe  for  cost  growth,  in 
the  case  of  the  MLRS,  the  strategy  was  successful.  The  technology  was  kept 
simple.  Even  though  the  desired  accuracy  was  not  obtained,  the  Army  has  been 
able  to  field  an  acceptable  system  relatively  quickly  to  fill  a  significant  need. 

2 .  The  competitive  prototyping  produced  systems  that  had  good  early  test  results. 

3 .  The  multiyear  contract  produced  real  cost  savings,  according  to  GAO,  and  also 
contributed  to  program  stability. 

4.  The  international  nature  of  the  program  may  have  contributed  to  the  political 
support  i:  received  and  to  its  financial  stability. 

1 1 .  Summary 

In  this  section,  we  examine  key  reasons  for  the  specific  program  outcomes 
observed  here.  We  group  programs  into  low,  medium,  and  high  categories  for 
development  schedule  growth  and  total  program  cost  growth. 

a.  Reasons  for  Schedule  Growth  in  Development 

Table  lll-l  summarizes  the  key  program  characteristics  that  led  to  low,  medium,  or 
high  schedule  growth  in  development.  Development  is  highlighted  here,  because  it  clearly 
represents  a  program  outcome — it  is  important  that  the  system  be  finished  on  time. 
Schedule  growth  in  production,  by  contrast,  is  often  a  policy  variable — e.g..  Congress 
decides  to  slow  the  program  for  budgetary  reasons. 
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Table  111-1.  Characteristics  of  Programs  With  Schedula  Growth  in  Davsiopment 


Two  of  the  three  programs  with  low  development  schedule  growth — the  TOW  2 
and  the  Sidewinder  A1M-9M — were  modification  programs  with  fairly  simple 
technologies.  The  other  program,  the  MLRS,  was  urgent  and  made  a  technical  tradeoff  to 
meet  the  schedule.  Programs  with  high  development  schedule  growth  included  two  joint- 
service  programs  (AIM-9L  and  AIM-7F)  and  two  programs  with  schedule  urgency  (AGM- 
65D/G  and  AIM-9L).  Underestimation  of  technical  difficulty  was  common.  In  two  cases 
(AIM-54A  and  AIM-9L),  development  was  slowed  tc  resolve  technical  problems,  in  the 
hope  that  they  would  not  spill  over  into  production.  In  the  case  of  the  Phoenix  AIM-54A, 
that  strategy  appeared  to  be  successful.  In  the  case  of  the  Sidewinder  AIM-9L,  we  cannot 
be  sure  that  the  high  production  cost  growth  was  due  to  development  problems  spilling 
over  into  production  because  the  AIM-9L  also  suffered  a  major  production  stretchout. 

b.  Reasons  for  Cost  Growth  (Total  Program) 

Table  III-2  summarizes  the  key  program  characteristics  that  led  to  cost  growth 
outcomes,  based  on  the  case  analyses.  As  in  the  previous  analysis,  programs  are  grouped 
into  low,  medium,  and  high  categories. 

All  four  programs  with  the  lowest  TPCG  are  characterized  by  low  stretch  in 
production.  Interestingly,  three  of  the  four  programs  in  this  group  were  also  characterized 
by  urgency.  In  the  case  of  the  MLRS,  the  requirement  was  modified  to  meet  a  deadline. 
(The  AIM-9M  may  be  something  of  an  anomaly  in  this  group,  because  it  was  intended  to 
fulfill  the  technical  goals  for  the  AIM-9L.) 

The  programs  with  high  TPCG,  by  contrast,  were  characterized  by  stretched 
production  schedules.  Both  AMRAAM  and  the  Phoenix  AIM-54C  had  high  levels  of 
concurrency  and  rushed  testing  programs.  Both  the  Phoenix  AIM-54C  and  the  Sidewinder 
AIM-9L  were  dual-sourced  for  technical  reasons  (e.g.,  to  gel  a  better  functioning  system) 
rather  than  principally  for  cost  savings.  In  the  case  of  the  AIM-54C,  funding  was  reduced 
to  move  more  quickly  to  the  next-generation  system,  and  five  years  were  spent  qualifying  a 
second  source  for  only  two  years  of  head-to-head  competition,  with  resulting 
inefficiencies.  AMRAAM  also  had  used  a  dual-sourcing  strategy  and  was  produced  at  less 
than  the  planned  rate — an  expensive  combination.  The  Sparrow  AIM-7M  also  suffered 
from  reduced  production  funding  in  order  to  fund  the  next-generation  system. 
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.  CiMracteristics  of  Program*  With  Cost  Growth  in  Total  Program  (eontinuad) 


C.  LESSONS  LEARNED  FROM  TACTICAL  AIRCRAFT  PROGRAMS 

1 .  Development 

Tactical  aircraft  outcomes  in  both  development  and  production  exhibited  less 
variability  than  the  outcomes  of  tactical  missiles.  Development  cost  growth  ranged  from  5 
to  S3  percent,  and  total  program  cost  growth  ranged  from  -9  to  40  percent.  In  part,  this 
may  be  because  our  sample  of  aircraft  is  smaller  than  the  sample  of  missiles.  However,  it  is 
also  the  case  that  development  programs  of  aircraft  appear  to  proceed  more  smoothly  than 
those  of  other  types  of  systems  [2  and  S]. 

Development  schedule  growth  was  small  in  the  aircraft  development  programs.  The 
highest  value  was  18  percent  for  the  F-14A. 

The  two  programs  with  the  lowest  development  cost  growth  are  the  F-5E  and  the 
F-15A/B.  The  F-5E  was  a  relatively  simple  development  program,  building  on  a 
commercial  system.  The  F-15  program  proceeded  smoothly,  with  relatively  few 
engineering  changes.  The  F-14A  had  the  highest  development  cost  growth,  53  percent. 
This  may  have  been  due  to  the  unusual  contracting  arrangement,  a  fixed-price  development 
contract  with  options  for  several  years  of  production.  Grumman  ran  into  cost  trouble  and 
insisted  on  changes  in  the  contract  before  it  would  produce  the  aircraft. 

2 .  Production 

Few  studies  have  analyzed  aircraft  production.  One  reason  may  be  that  “the 
production  phase  is  assumed  to  be  less  interesting  as  outcomes  at  this  point  are  more 
predictable  than  during  development,  and  the  problems  that  do  occur  are  generally  traceable 
to  faulty  decisions  made  earlier  in  the  program”  [6,  p.  159]. 

But  from  the  contractor’s  viewpoint,  production  opportunities  define  the  business 
environment  contractors  face.  A  contractor  cannot  keep  going  on  developments  alone. 
According  to  Mayer,  “without  existing  or  potential  production  work,  contractors  scale  back 
investment,  reduce  staffing,  and  redirect  efforts  toward  other  markets  that  have  a  brighter 
outlook.” 

As  compared  with  the  1950s,  the  1990s  have  fewer  new  production  starts,  fewer 
production  lines  open  at  any  given  time,  and  longer  production  runs,  but  with  smaller  total 
quantity — i.e.,  a  lower  production  rate.  None  of  these  trends,  of  course,  is  at  all  favorable 
to  reducing  costs. 
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During  the  1960s,  we  saw  a  trend  toward  fewer  new  production  starts.  However, 
the  Vietnam  War  greatly  increased  the  demand  for  aircraft,  and  production  buys  were  quite 
large.  The  space  program  also  helped  to  keep  those  contractors  who  were  frozen  out  of  the 
new  airframe  business  going.  In  the  1970s.  several  new  programs  started,  but  annual  buys 
were  reduced  after  the  war  ended.  In  the  198()s.  mans  new  programs  started,  but  not  all  of 
them  survived.  The  1990s  will  be  a  decade  where  on!\  u  few  prime  contractors  for  tactical 
aircraft  will  survive. 

The  A-X  program  has  already  been  canceled,  and  it  appears  that  the  F-22  and  the 
F/A-18E/F  may  be  the  only  tactical  aircraft  programs  in  production  by  the  end  of  the 
decade.  Several  systems  currently  in  production — the  F-t4,  the  F-15,  and  the  F-16 — are 
expected  to  end  within  the  next  few  years. 

Nevertheless,  one  characteristic  of  the  production  environment  is  that  production 
lines  tend  to  stay  active  much  longer,  even  as  the  total  number  of  units  produced  declines. 
Table  III-3  shows  the  average  length  of  production  runs  for  fighter  and  attack  programs. 
Continuing  these  trends  and  combining  it  with  a  trend  toward  fewer  new  programs 
provides  further  weight  to  the  conclusion  that  fewer  units  will  be  produced  per  year  and 
that  contractors  who  fail  to  participate  in  one  of  the  few  new  programs  will  no  longer  be 
prime  contractors.  A  model  cited  in  Reference  [6]  projected  that  a  company  would  lose  its 
viability  as  a  prime  contractor  after  only  18  months  without  a  contract,  after  which  it  would 
be  forced  to  focus  on  subcontracting. 

Table  111*3.  Average  Length  of  Production  Runs  for 
Air  Force  and  Navy  Fighters 


Years 

Length  (Years) 

1951-1960 

9.4“ 

1961-1970 

20.0 

1971-1980 

18.0 

Source:  Reference  [6], 

®  Excludes  the  F-5  Tighter,  which  was  built  primarily 

for  export. 

In  our  sample  of  tactical  aircraft,  there  was  considerable  variation  in  production 
schedule  stretch.  The  F-5E  program,  for  example,  produced  its  planned  quantity  30  percent 
faster  than  planned,  while  the  F-14A  and  AV-8B  programs  were  stretched  out,  producing 
at  about  half  the  planned  rate. 

With  respect  to  cost,  the  F-5E  came  in  21  percent  under  expected  cost,  benefiting 
from  a  high  production  rate  and  technical  simplicity.  However,  both  the  AV-8B  and  the 
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F-14A  suffered  from  production  stretch,  yet  exhibited  low  production  cost  growth.  In  the 
case  of  the  F-14A,  slower  production  was  accompanied  by  funding  cuts.  The  original  plan, 
to  produce  463  aircraft  in  5.5  years,  was  recognized  early  as  unrealistic,  and  the  contractor 
had  plenty  of  time  to  scale  back  production  facilities.  Moreover,  the  development  of  the 
F-14D  aircraft  occurred  at  about  the  same  time,  and  may  have  helped  the  contractor  cope 
with  costs  on  the  F-14A.  In  the  case  of  the  AV-8B.  the  program  never  produced  as  many 
aircraft  as  planned.  However,  the  presence  of  other,  larger  programs  in  the  same  plant — 
including  the  F/A-18  and  the  F-15 — may  have  helped  to  contain  costs. 

The  program  with  the  highest  production  cost  growth  is  the  F/A-18,  which 
exhibited  production  cost  growth  of  42  percent,  despite  little  production  stretch.  The 
F/A-18  originated  as  the  losing  entry  in  a  competition  to  provide  a  lightv/eight  fighter  for 
the  Air  Force  and  the  Navy.  The  Air  Force  selected  the  YF- 1 6  in  a  prototype  flyoff  over  the 
YF-17.  Congress  encouraged  the  Navy  to  adapt  the  YF-16  to  its  special  needs.  However, 
the  Navy  redesignated  the  YF-17  as  the  F-18.  Technical  problems  in  range,  cycle  time, 
strafing,  roll,  and  faulty  radar  images  were  identified  at  the  DSARC  lU  meeting.  Despite 
claims  that  the  digital  flight  control  system  would  eliminate  the  need  to  physically  change 
the  test  aircraft  (because  changes  would  be  made  in  the  software),  the  Navy  and  the 
contractor  made  more  physical  changes  to  the  F-18  than  to  any  other  fighter  aircraft  in  the 
last  twenty  years  [12].  Technical  changes,  panicularly  late  in  development  or  in  production, 
are  expensive. 
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IV.  COST/SCHEDULE  RELATIONSHIPS 


A.  INTRODUCTION 

This  chapter  describes  the  regression  analyses  used  to  illuminate  the  relationship 
between  cost  and  schedule  growth  in  ongoing  acquisition  programs.  The  relationships  can 
be  used  to  project  the  effect  of  schedule  changes  on  cost.  Relationships  are  presented  for 
development,  production,  and  the  total  program  for  tactical  missiles  and  tactical  aircraft. 

B.  METHODS 

We  used  several  analytical  tools  to  construct  development  equations  for  this  diverse 
group  of  systems,  including  linear  and  log-linear  regression,  weighted  regression,  and 
correlation  analysis.  Examining  the  relationship  between  cost  and  schedule  growth  in 
development,  we  concluded  that  it  was  not  appropriate  to  consider  the  major  independent 
variable,  DSG,  to  be  exogenous.  Other  variables  determine  DSG.  Therefore,  we  estimated 
the  tactical  missile  development  relationship  as  a  simultaneous  system  of  equations. 

We  considered  the  possibility  that  the  relationship  between  development  cost 
growth  and  development  schedule  growth  may  not  be  linear.  We  expect  that,  as  the 
development  schedule  stretches  beyond  plan,  development  costs  would  increase.  However, 
we  also  expect  that  compressing  the  development  schedule  would  increase  costs.  Thus,  the 
true  relationship  is  probably  U-shaped.  However,  in  our  tactical  missile  data  set,  the  only 
system  that  had  a  compressed  development  schedule  was  the  Pershing  2,  where  a 
17-percent  speedup  was  accompanied  by  13  percent  cost  growth.  For  the  tactical  aircraft, 
the  data  set  was  judged  to  be  too  small  to  do  more  than  single-equation  estimation. 

The  dependent  variables  were  the  cost  growth  measures  described  in  Chapter  II. 
The  candidate  independent  variables,  along  with  the  areas  in  which  they  are  used,  are  listed 
in  Table  IV- 1.  The  variables  include  several  variants  of  schedule.  (These  variables  are 
defined  more  fully  in  Chapter  II.)  They  dso  include  some  variables  relating  to  program 
management,  including  program  characteristics  and  acquisition  initiatives  (e.g.,  design-to- 
cost,  multiyear  procurement,  and  total  package  procurement).  In  addition,  variables 
describing  the  size  of  the  program  in  terms  of  planned  cost  were  used.  Data  for  the  non¬ 
schedule  supplemental  variables  are  provided  in  Appendix  A. 
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Table  IV-1.  Candidate  Independent  Variables 
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TaW*  IV-1.  Candidate  IndapMtdant  Variablat  (continued) 


Equation  formulations  that  appeared  on  the  surface  to  be  successful  sometimes  had 
to  be  discarded  because  of  high  correlations  of  the  independent  variables.  For  example, 
several  of  the  acquisition  initiatives  were  highly  correlated  with  various  schedule  variables. 
Because  our  purpose  here  was  to  develop  cost/schedule  relationships,  we  generally 
dropped  the  initiative  variables  and  left  the  schedule  variables  in  the  equation  when  choices 
had  to  be  made.  The  only  initiative  variable  retained  was  a  multiyear  procurement  variable 
in  the  equation  for  tactical  missile  production  cost  growth.  The  initiatives  may  still  be 
important  influences  on  acquisition  costs  and  schedules. 

C.  TACTICAL  MISSILES 

1 .  Development 

This  section  discusses  the  development  relationships  for  tactical  missiles.  The 
development  cost  growth  database  consisted  of  the  twenty  programs  whose  outcomes  are 
displayed  in  Chapter  II.  The  dependent  variable  was  the  development  cost  growth  ratio  for 
spending  from  Milestone  II  to  the  end  of  development  of  the  first  version,  DCGM2. 

As  previously  indicated,  lack  of  data  prevented  us  from  examining  the  case  of  a 
development  schedule  more  compressed  than  planned,  :>ince  there  was  only  one  such  case 
in  the  data  set.  If  data  were  sufficient,  we  could  allow  for  this  effect  by  using  a  quadratic 
term  or  a  non-linear  specification.  However,  linear  specifications  best  depicted  this  data  set. 

Because  the  DSG  and  DCG  equations  comprised  a  simultaneous  system,  we  used 
the  two-stage  least  squares  method  of  estimation.  (Detailed  descriptions  of  the  method  can 
be  found  in  econometrics  texts  such  as  [13]  and  [14].)  The  first  equation  was  estimated 
using  ordinary  least  squares,  and  the  predicted  value  of  DSG  from  Equation  1  (DSGHAT) 
was  calculated  for  each  observation.  Then,  DSGHAT  was  used  as  the  independent  variable 
in  Equation  2. 

DSG  =  1.6325  -  (.0095  x  PDS)  +  (.2056  x  DQG)  (.4038  x  IMD)  +  (.7128  x  IIRMD)  (1) 
(.02)  (.05)  (.04)  (.07) 

Adjusted  *  .578  SEE  =  .344 

Numbers  in  parentheses  below  the  coefficients  are  significance  levels.  Adjusted  is  the 
percentage  of  variation  in  DSG  explained  by  the  model,  adjusted  to  reflect  the  number  of 
independent  variables,  and  SEE  is  the  standard  error  of  the  estimate. 
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Equation  1  contains  the  following  variables: 

•  PDS,  planned  development  schedule.  The  lower  this  estimate  (the  more 
schedule  optimism),  the  higher  the  DSG  ratio.  This  variable  is  statistically 
significant  at  the  .02  level. 

•  DQG,  development  quantity  growth.  Having  to  build  unplanned  development 
items  is  an  indicator  of  technical  problems  and  impending  schedule  growth. 
The  coefficient  is  positive  and  statistically  significant  at  the  .05  level. 

•  IMD,  the  air  intercept  dummy  variable,  is  positive  and  significant  at  the  .04 
level.  Notably,  the  two  systems  with  the  highest  DSG,  the  AIM-7F  and  the 
AIM-9L,  are  both  air  intercept  missiles. 

•  IIRMD,  the  HR  Maverick  dummy,  is  positive  and  significant  at  the  .07  level. 
This  variable  is  included  to  remove  the  effect  of  the  HR  Maverick,  which  is 
atypical  of  the  rest  of  the  data.  It  has  an  unusual  administrative  delay  in  FSD 
that  did  not  appear  to  be  due  to  technical  problems. 

Using  the  coefficients  from  Equation  1,  predicted  values  for  DSG,  dubbed 
DSGHAT,  wer :  obtained.  This  variable  was  used  as  an  independent  variable  in  the  DCG 
equation  (Equation  2),  along  with  IIRMD. 

DCG  =  -1.366  -  (1.5972  x  IIRMD)  +  (2.034  x  DSGHAT)  (2) 

(.03)  (.0001) 

Adjusted  R^  =  .63 1  SEE  =  .638 

The  development  cost  growth  model  in  Equation  3  has  DSG  as  the  dependent 
variable.  Equation  4  has  DCG  as  the  dependent  variable.  These  equations  can  be  used  to 
predict  development  cost  growth  in  ongoing  acquisition  programs. 

Plots  ot  actual  versus  fitted  values  are  included  as  Figure  IV- 1  (DSG  model)  and 
Figure  IV-2  (DCG  model).  In  each  case,  a  45-degree  reference  line  indicates  the  points  at 
which  actual  and  fitted  values  would  be  equal.  Points  above  the  line  were  overestimated; 
points  below  the  line  were  underestimated. 

The  simultaneous  model  can  be  used  to  provide  estimates,  both  at  Milestone  II  and 
beyond,  of  schedule  and  cost  growth.  Suppose  that  we  are  trying  to  project  the  outcome  of 
an  air  intercept  missile  program  with  an  estimated  schedule  of  80  months.  Therefore, 
PDS  s  80  and  IMD  =  1 .  Because  we  have  no  reason  to  think  that  the  development  quantity 
estimate  is  incorrect,  we  set  DQG  =  1.  Because  we  do  not  expect  an  arbitrary  schedule  hold 
similar  to  the  HR  Maverick,  we  set  IIRMD  =  0.  This  yields  a  schedule  growth  estimate  of 
1.4819,  or  48  percent  schedule  growth.  To  project  cost  growth,  set  IIRMD  =  0  and 
DSGHAT  =  1.4819,  and  calculate  the  value  from  Equation  2.  This  indicates  potential 
development  cost  growth  of  1 .6482. 
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Figure  IV'2.  Dvva'opmtnt  Cost  Growth  for  Tactical  Missilas 

As  the  program  progresses,  the  value  of  DQG  may  change.  Suppose  that  during 
EMD  the  program  has  to  build  10  percent  more  development  items  than  planned.  Then 
DQG  becomes  1.10,  and  the  values  of  DSG  and  DCG  can  be  recalculated.  The  new 
projected  schedule  growth  is  50  percent,  and  the  new  projected  cost  growth  is  69  percent. 
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To  supplement  the  simultaneous  model,  we  also  include  a  simple,  single-equation 
model  that  estimates  DCG  as  a  function  of  DSG  and  DQG: 

DCG  =  •  .1427  +  (.3776  x  DSG)  +  (.9094  x  DQG)  (3) 

(.04)  (.0001) 


Adjusted  =  .920 


SEE  =  .303 


Equation  3  can  be  used  to  predict  development  cost  growth  as  the  program 
progresses.  The  fit  of  the  equation  is  shown  in  Figure  IV-3. 
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Figure  IV*3.  Single  Equation  Model  for 
Development  Coat  Growth  for  Tactical  Miseiies 


2.  Production  Relationship  for  Tactical  Missiles 

The  best-fitting  equation  (Equation  4)  includes  production  stretch  positively,  as 
expected.  Planned  unit  cost,  a  proxy  for  complexity,  also  enters  positively.  The  programs 
with  multiyear  procurement  (MLRS,  TOW,  TOW  2,  Patriot,  Improved  Hawk,  and 
Shillelagh)  had  significantly  lower  production  cost  growth.  Examination  of  this  equation 
indicated  no  problems  with  muiticoliinearity  or  with  individual  cases  driving  the 
relationship. 

PCG  =  1.1 1  +  (.2858  X  PSS)  +  (.2321  x  PUC)  -  (.3124  x  MYP)  (4) 

(.006)  (.03)  (.07) 


Adjusted  R^  =  .458 


SEE  =  .298 


(4) 


Numbei*s  in  parentheses  below  the  coefficients  are  significance  levels.  Adjusted  is  the 
percentage  of  cost  growth  variation  explained  by  the  model,  adjusted  to  reflect  the  number 
of  independent  variables,  and  SEE  is  the  standard  error  of  the  estimate. 

The  fit  of  the  equation  is  shown  in  Figure  IV -4. 


Predicted  PCG 

Figure  iV«4.  Production  Cost  Growth  for  Tacticai  Missiles 

An  alternative  equation  (Equation  5)  contains  concurrency  as  a  measure.  While  it  is 
not  statistically  significant,  concurrency  was  found  in  the  case  analyses  to  be  an  important 
contributor  to  cost  growth.  The  equation  also  uses  development  cost  growth  as  an 
independent  variable.  Programs  that  have  had  high  development  cost  growth  are  mere 
likely  to  have  high  production  cost  growth.  Equation  5  could  be  used  as  a  check  of  the 
other  equation. 


PCG  =  .9512  +  (.2143  x  PSS)  +  (.1199  x  DCG)  +  (.4147  xCONC)  (5) 

(.08)  (.07)  (.12) 

Adjusted  =  .344  SEE  =  .236 

To  use  the  production  cost  growth  equations,  consider  a  new  system  with  planned 
unit  production  cost  of  $500,000  ( $0.5  million  in  FY  1994  dollars),  no  production  stretch 
expected,  and  no  multiyear  procurement.  Set  PSS  =  1,  PUC  =  .5,  and  MYP  =  0.  The 
production  cost  growth  projection  from  Equation  4  is  1.51.  To  use  Equation  5,  additional 
data  are  necessary.  Suppose  that  the  program  is  expected  to  have  SO  percent  cost  growth  in 
development  and  50  percent  concurrency.  Then  DCG  =  1 .5  and  CONC  =  .5.  This  results 
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in  a  cost  growth  projection  of  1.55.  As  the  program  goes  on,  suppose  that  schedule  stretch 
of  50  percent  occurs.  For  example,  the  production  schedule  could  grow  50  percent  with  no 
corresponding  increase  in  production  quantity.  Alternatively,  the  quantity  could  be  cut  by 
one-third  with  no  corresponding  decrease  in  schedule.  Setting  PSS  =  1.5  yields  different 
projections  of  production  cost  growth:  1.65  from  Equation  1  and  1.66  from  Equation  2. 

3 .  Total  Program  Cost  Growth  for  Tactical  Missiles 

In  developing  the  total  program  relationship  for  tactical  missiles,  we  included  the 
same  candidate  variables  as  were  used  in  production.  We  added  total  schedule  growth 
(TSG).  The  best  equation  is: 

TPCG  =  .7645  +  (.3677  x  TSG)  +  (.1845  x  PUC)  +  (.2729  x  IMD)  (6) 

(.005)  (.04)  (.04) 

Adjusted  =  .500  SEE  =  .259 

Numbers  in  parentheses  below  the  coefficients  are  significance  levels.  Adjusted  is  the 
percentage  of  cost  growth  variation  explained  by  the  model,  adjusted  to  reflect  the  number 
of  independent  variables,  and  SEE  is  the  standard  error  of  the  estimate. 

The  fit  of  this  equation  is  good,  as  seen  in  Figure  rV-5,  and  signs  of  the  coefficients  are 
reasonable.  As  seen  in  both  development  and  production,  schedule  growth  is  associated 
with  cost  growth.  There  are  no  apparent  problems  with  multicollinearity  or  with  individual 
cases  driving  the  relationship. 


Figure  (V-5.  Total  Program  Coat  Growth  for  Tactical  Miaslles 
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D.  AIRCRAFT 


1 .  Development  Cost  Growth 

The  aircraft  development  regression  is  different  from  that  for  the  tactical  missiles. 
Aircraft  programs  typically  receive  more  management  attention  than  tactical  missiles 
programs  and  they  are  larger  in  terms  of  cost.  Moreover,  perhaps  because  of  the 
widespread  use  of  contractor  labor  across  the  United  States,  they  are  unlikely  to  be  allowed 
to  fail. 

As  previously  noted,  development  cost  growth  for  aircraft  is  lower  in  percentage 
terms  and  shows  less  variation  than  for  tactical  missiles.  The  raw  relationship  between 
development  cost  growth  and  development  schedule  growth  is  not  strong.  However,  a 
regression  weighted  by  planned  development  cost  (in  millions  of  FY  1994  dollars) 
provided  reasonable  results.  The  regression  also  includes  a  dummy  variable  for  the  AV-8B 
(AV8BD).  The  AV-8B  is  distinctive  in  two  ways.  It  is  the  only  modification  program 
among  the  seven  tactical  aircraft  in  the  database.  It  also  had  a  slow  ramp-up  to  full-rate 
production  [15].  In  all  program  phases,  the  relationship  of  cost  growth  to  schedule  growth 
in  the  AV-8B  is  atypical.  In  development,  the  AV-8B  had  schedule  growth  of  only  3 
percent  but  cost  growth  of  40  percent.  Because  we  cannot  be  certain  that  the  effect  we  are 
observing  is  common  to  all  tactical  aircraft  modification  programs,  or  is  distinctive  to  the 
AV-8B,  we  have  labeled  the  variable  AV8BD  rather  than  MOD.  The  development 
cost/schedule  relationship  for  tactical  aircraft  is: 

DCG  =  .-1.135  +  (2.1614  x  DSG)  +  (.3514  x  AV8BD)  (7) 

(.0006)  (.01) 

Adjusted  r2  =  .860  SEE  =  2.86 

Numbers  in  parentheses  below  the  coefficients  are  significance  levels.  Adjusted  is  the 
percentage  of  cost  growth  variation  explained  by  the  model,  adjusted  to  reflect  the  number 
of  independent  variables,  and  SEE  is  the  standard  error  of  the  estimate. 

Regression  was  weighted  by  planned  development  cost  in  FY  1994  dollars.  A 
scatter  plot  of  DSG  and  DCG  showing  the  fit  of  the  equation  is  shown  in  Figure  IV-6. 
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Figure  IV>  6.  Development  Coet  Growth  for  Tactical  Aircraft 

2.  Production  Cost  Growth 

In  the  case  of  production  cost  growth,  the  strongest  relationship  was  a  logarithmic 
one.  The  regression  was  run  in  linear  form  on  the  logarithms  of  the  variables.  The 
dependent  variable  was  the  log  of  the  production  cost  growth  ratio,  LNPCG.  The 
independent  variables  included  the  log  of  the  number  of  months  required  to  produce  the 
planned  quantity,  LNAPS,  and  the  AV8BD  variable. 

The  equation  is  shown  here  transformed  back  to  arithmetic  space.  The  constant  term 
includes  a  correction  to  account  for  the  bias  of  the  logarithmic  form,  as  does  the 
statistic.  This  process  is  described  in  Appendix  B.  In  arithmetic  space,  the  AV-8B  dummy 
variable  changes  its  form.  It  takes  the  value  e  for  the  AV-8B  and  1  for  all  other  aircraft.  To 
avoid  confusion  with  AV8BD,  we  have  called  this  variable  EAV8B.  The  arithmetic 
transformation  of  the  equation  is: 

PCG  =  .2678  X  APS  -3286  y  EAV8B  (8) 

(.004)  (.02) 

Adjusted  =  .845  SEE  =  .086 

Numbers  in  parentheses  below  the  coefficients  are  significance  levels.  Adjusted  R^  is  the 
percentage  of  cost  growth  variation  explained  by  the  model,  adjusted  to  reflect  the  number 
of  independent  variables,  and  SEE  is  the  standard  error  of  the  estimate. 
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Figure  IV-7  is  a  scatter  plot  of  PCG  and  APS  showing  the  fit  of  the  equation.  The 
equation  has  a  reasonable  standard  error.  Again,  the  AV-8B’s  unusually  slow  ramp-up  did 
not  have  as  large  a  cost  penalty  as  was  typical  of  the  rest  of  the  group. 


Figure  IV-7.  Production  Coet  Growth  for  Tactical  Aircraft 


3.  Total  Program  Cost  Growth 

We  also  estimated  a  total  program  relationship  for  tactical  aircraft.  As  with 
production,  the  preferred  relationship  includes  a  measure  of  total  schedule  rather  than 
schedule  growth  and  also  includes  a  dummy  variable  for  the  AV-8B.  Also,  similar  to  the 
production  equation,  the  total  program  equation  is  in  logarithmic  form.  It  was  estimated 
using  the  logarithms  of  the  variables,  then  transformed  back  into  arithmetic  space.  It 
includes  a  corrected  constant  term  to  account  for  the  bias  of  the  logarithmic  equation  form. 
As  estimated,  the  dependent  variable,  LNTPCG,  was  the  natural  log  of  the  total  program 
cost  growth  ratio,  and  the  independent  variables  included  the  natural  log  of  the  total 
schedule  in  months  and  AV8BD. 

As  with  the  production  relationship,  the  total  program  cost  growth  equation  is 
shown  here  transformed  back  to  arithmetic  .space.  The  constant  term  includes  a  correction 
to  account  for  the  bias  of  the  logarithmic  form.  In  arithmetic  space,  the  AV-8B  dummy 
variable  changes  its  form.  It  takes  the  value  e  for  the  AV-8B  and  1  for  all  other  aircraft.  To 
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avoid  confusion  with  AV8BD,  we  have  called  this  variable  EAV8B.  The  arithmetic 
transformation  of  the  equation  is: 

TPCG  =  .3785  X  ATS  -^365  ^  EAV8B  (9) 

(.003)  (.006) 

Adjusted  =  .890  SEE  =  .053 

Numbers  in  parentheses  below  the  coefficients  are  significance  levels.  Adjusted  is  the 
percentage  of  variation  in  TPCG  explained  by  the  model,  adjusted  to  reflect  the  number  of 
independent  variables,  and  SEE  is  the  standard  error  of  the  estimate. 

Figure  rV-8  is  a  scatter  plot  of  TPCG  and  ATS  showing  the  fit  of  the  equation. 

1.6 
1.4 
1.2 

§  ^ 

^  0.8 

0.6 

0.4 

0.2 

0 

0  25  SO  75  100  125  ISO  175  200  225 

ATS 

Figure  IV<8.  Total  Program  Cost  Growth  for  Tactical  Aircraft 

E.  DISCUSSION  AND  SUMMARY 

Using  standard  techniques,  we  were  able  to  estimate  regressions  that  describe  the 
relationship  between  cost  and  schedule  for  tactical  missiles  and  tactical  aircraft.  We  were 
able  to  derive  separate  equations  for  development  and  production,  as  well  as  an  overall 
equation  relating  total  program  cost  growth  to  the  program  schedule. 

The  tactical  missile  regressions  generally  fit  well,  with  intuitively  correct  signs. 
They  can  reasonably  be  used  for  getting  a  sense  of  the  cost  implications  of  schedule 
changes.  The  only  exception  is  that  the  development  relationship  should  not  be  used  for 
predicting  the  cost  implications  of  significantly  speeding  up  development.  The  tactical 
missile  relationships  indicate  the  feasibility  of  the  approach  and  the  usability  of  the 
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projection  tools.  However,  in  the  tactical  aircraft,  it  is  difficult  to  make  inferences  based  on 
only  seven  observations.  Additional  data  development  work  is  needed  to  make  the  aircraft 
regressions  into  practical  projection  tools. 
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V.  SUMMARY  AND  CONCLUSIONS 


This  paper  reports  on  the  beginning  of  an  effort  to  describe  cost  and  schedule 
growth  patterns  associated  with  the  acquisition  of  major  systems,  to  identify  reasons  for  the 
growth,  and  to  develop  a  way  to  anticipate  likely  growth  in  development  and  early 
production  phases.  By  looking  at  past  acquisition  program.s,  we  examined  what  separates 
the  kinds  of  programs  completed  on  schedule  and  within  cost  plans  from  those  that 
experience  cost  and  schedule  growth. 

Tactical  missile  programs  were  selected  for  a  pilot  effort,  largely  on  the  basis  of 
data  availability.  Cost  and  schedule  growth  were  measured  in  development  and  in 
production.  These  measures  showed  a  great  deal  of  variability  among  the  20  programs 
examined.  Programs  took  from  SO  months  to  137  months  from  Milestone  II  to  IOC.  Only 
two  of  the  tactical  missile  programs  were  finished  on  time.  The  program  with  the  highest 
development  schedule  growth  exceeded  its  plan  by  180  percent.  Two  programs  came  in 
under  budget,  while  two  others  doubled  in  cost  from  their  Milestone  D  plaji. 

Selected  tactical  missile  programs  were  examined  in  more  detail  to  determine  the 
reasons  for  schedule  and  cost  growth.  Keys  to  preventing  schedule  growth  in  development 
are  technictil  realism  and  willingness  to  make  tradeoffs.  Programs  with  high  development 
schedule  growth  tended  to  underestimate  technical  difficulty.  Two  of  the  five  programs 
with  high  DSG  also  had  high  overall  cost  growth.  However,  in  three  of  the  five  cases  of 
high  DSG,  it  appeared  that  a  strictly  phased  approach — resolving  problems  in  development 
when  spending  levels  are  low — resulted  in  lower  levels  of  overall  cost  growth.  Keys  to 
preventing  overall  cost  growth  are  correctly  estimating  the  degree  of  technical  difficulty  in 
the  programs  and  maintaining  the  planned  production  schedule.  Programs  that  employed  a 
high  degree  of  concurrency,  that  had  to  be  dual-sourced  for  technical  reasons  or  that  were 
dual-sourced  at  less  than  full  rate  had  high  cost  growth.  In  one  case,  the  threat  of 
competition  appeared  to  reduce  costs. 

There  were  enough  common  factors  in  the  tactical  missiles  to  suggest  that 
estimating  quantitative  relationships  would  be  possible.  The  equations  presented  here  relate 
cost  growth  to  schedule  growth  in  development  and  production.  In  development,  a 
simultaneous  model  links  an  estimating  relationship  for  schedule  growth  with  a  cost  growth 
equation.  The  major  determinant  of  development  schedule  growth  was  quantity  growth  (the 
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need  to  produce  more  items  for  testing  than  planned).  Other  variables  in  the  DSC  equation 
were  the  planned  schedule  for  the  program  and  dummy  variables  for  intercept  missiles  and 
for  one  outlier.  The  results  of  this  equation  feed  into  a  DCG  equation.  Cost  growth  in 
production  was  linked  to  schedule  stretch,  planned  unit  cost  (a  proxy  for  complexity),  and 
multiyear  procurement.  These  equations  can  be  used  to  identify  programs  likely  to 
experience  future  growth,  as  well  as  to  monitor  the  effect  of  schedule  changes  on  cost 
growth.  The  relationships  can  be  used  to  identify  which  programs  require  more  detailed 
examination. 

Cost  and  schedule  growth  measures  were  also  calculated  for  a  sample  of  seven 
tactical  aircraft.  These  measures  are  less  dispersed  than  those  for  tactical  missiles.  The 
aircraft  programs  tended  to  receive  more  management  attention  and  more  protection  from 
schedule  stretch  than  the  tactical  missiles.  The  highest  cost  growth  index  was  1.40,  versus 
2.23  for  the  tactical  missiles.  Quantitative  relationships  were  also  developed  for  this  group, 
but  the  small  sample  size  requires  that  they  be  regarded  as  tentative.  This  exercise  indicated 
that  the  cost/schedule  relationships  look  different  for  different  equipment  types. 

In  the  light  of  these  results,  we  conclude  that  DoD  and  other  personnel  who  review 
acquisition  programs  would  benefit  from  a  review  method  based  on  detailed  information 
about  the  strategies  and  outcomes  of  past  programs.  This  is  the  first  step  for  development 
of  such  a  review  method  that  examines  the  reasonableness  of  program  plans  and  assesses 
the  effect  on  cost  of  schedule  changes. 

The  case  analyses  of  tactical  missiles  also  indicate  that  many  useful  lessons  can  be 
gleaned  from  historical  perspective,  lessons  that  cannot  always  be  captured  in  a  quantitative 
estimating  relationship.  At  the  beginning  of  a  weapon  system  development  program,  the 
feeling  often  prevails  that  the  program  is  unique  and  will  somehow  avoid  repeating  the 
problems  of  past  programs.  Yet  cost  and  schedule  growth  persist.  Despite  individual 
differences  in  programs,  the  importance  of  understanding  the  level  of  technical  difficulty 
when  original  schedule  and  cost  estimates  are  made,  of  suict  phasing  and  vigorous  testing, 
and  of  adhering  to  production  plans  are  borne  out  by  analysis  of  past  strategies  and 
outcomes. 
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DATA  USED  IN  REGRESSION  ANALYSES 


3.628  1.498  449  S24 

15.204  2.412  1.417  4.806 

1.21  0.86  0.06  I  0.01 


Table  A-2.  Tactical  Aircraft  Data 
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STATISTICAL  CORRECTION  FOR  BIAS  IN 
LOGARTTHMIC  SPECIFICATIONS  OF  EQUATIONS  FOR 
TACTICAL  AIRCRAFT  PRODUCTION  AND 
TOTAL  PROGRAM  COST  GROWTH 


APPENDIX  B. 

STATISTICAL  CORRECTION  FOR  BIAS  IN 
LOGARITHMIC  SPECIFICATIONS  OF  EQUATIONS  FOR 
TACTICAL  AIRCRAFT  PRODUCTION  AND 
TOTAL  PROGRAM  COST  GROWTH 

The  cost/schedule  relationships  for  tactical  aircraft  production  and  total  program 
costs  employed  logarithmic  specifications.  These  relationships  had  the  final  forms: 

PCG  =  .2678  X  APS  ^“6  eaV8B  ' 

(.004)  (.02) 

Adjusted  =  .845 


TPCG  *  .3785  X  ATS  x  EAV8B  -  3262 
(.003)  (.006) 

Adjusted  *  .890 

as  indicated  in  Chapter  IV.  The  purpose  of  this  appendix  is  to  show  how  these  final  forms 
were  derived. 

These  relationships  have  the  generic  f<vm: 

y  =  aoxX,««xX2«* 

where  T  is  the  dependent  variable  and  X,  is  an  independent  variable.  X2  is  an  independent 

dummy  variable  set  equal  to  e  for  the  observation  of  interest  (in  this  case  the  AV-8B)  and  1 
otherwise.  The  quantities  Oq,  a,,  and  02  are  parameters  to  be  estimated. 

To  estimate  the  parameters,  analysts  typically  linearize  by  taking  natural  logarithms 
of  both  sides  of  the  equation  and  adding  an  error  term  u,  giving: 

Inf  -  <2q'  fli  X  lnX|  +  02  ^  lnX2  +  u. 

The  dummy  variable  InXj  is  thus  in  the  familiar  0/1  form,  and  the  equation  to  be 
estimated  is  linear.  Note  that  a^  and  02  retain  the  same  form,  and  estimates  from  the  linear 

regression  can  be  substituted  directly  into  the  multiplicative  equation.  However,  the 
constant  term  Oq  changes  form — it  becomes  Oq',  or  Ihoq.  Given  an  estimate  of  Qq  from  the 
linear  regression,  the  logical  estimator  of  Qq  is  e^\  This  estimiue  of  the  original  constant 
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term  Aq  inherits  the  desirable  asymptotic  properties  from  the  linear  regression,  including 
consistency  and  asymptotic  efficiency.  It  does  not,  however,  inherit  the  small  sample 
properties,  particularly  unbiasedness.  (Unbiasedness  is  desirable,  because  it  ensures  that 
the  expectation  of  the  mean  of  is  equal  to  a„.) 

To  correct  for  the  bias  in  the  estimator  <* “" .  we  applied  a  smearing  estimate 
correction  factor  recommended  in  [16 J.  The  estimate  of  the  adjusted  coefficient  of 
determination  (R^)  was  also  corrected.  The  procedure  is: 

1 .  Calculate  the  unadjusted  predicted  values  of  Y  using  the  estimates  from  the 
linear  regression — a,,  a.^,  and  e^. 

2 .  Calculate  the  actual/predicted  ratio  for  each  observation  and  find  its  mean. 

3.  Multiply  e'^o  by  the  factor  calculated  in  step  2.  The  result  is  the  adjusted 
constant  term. 

4.  Using  the  adjusted  constant  term,  re-calculate  predicted  values  for  Y,  the  error 
sum  of  squares  (ESS),  and  the  total  sum  of  squares  (TSS). 

5.  Calculate: 

R2  =  l-fESSA-SS) 
and 

Adjusted  R^  s  1-  ((7-  DAT  -  IC)]0-  R^) 
where  7  is  the  number  of  observations  and  K  is  the  number  of  independent  variables. 

The  better  the  fit,  the  less  difference  the  adjustment  makes.  In  our  equations,  the 
adjustment  made  a  difference  of  less  than  1  percent  in  the  constant  term  and  less  than  3 
percentage  points  in  the  adjusted  R^.  The  unadjusted  equations  were: 

PCG  =  .2672  X  APS  x  EAV8B  * 

(.004)  (.02) 

Adjusted  R^  =  .870 
and 

TPCG  =  .3782  X  ATS  ^365  ^  eaV8B  ' 

(.003)  (.006) 

Adjusted  R^  =  .893 
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ABBREVIATIONS 


ABBREVIATIONS 


AAAM 

Advanced  Air-to-Air  Missile 

AMRAAM 

Advanced  Medium-Range  Air-to-Air  Missile 

DAB 

Defense  Acquisition  Board 

DoD 

Department  of  Defense 

DSARC 

Defense  Systems  Acquisition  Review  Council 

DSG 

development  schedule  growth 

DT&E 

development  test  and  evaluation 

ECCM 

electronic  counter-countermeasures 

FLIR 

forward-looking  infrared 

FOT&E 

follow-on  test  and  evaluation 

FSD 

full-scale  development 

FSED 

full-scale  engineering  developtiwnt 

GAO 

General  Accounting  Office 

HARM 

High-Speed  Anti-Radiation  Missile 

HOMS 

Hellfue  Optimized  Missile  System 

IDA 

Institute  for  Defense  Analyses 

UR 

imaging  infrared 

IOC 

initial  operational  c^ability 

lOT&E 

initial  operational  test  and  evaluation 

MOU 

memorandum  of  understanding 

MIBMA 

mean  time  between  maintenam^  actions 

NATO 

North  Atlantic  Treaty  Organization 

OPEVAL 

operational  evaluation 

OSD 

Office  of  the  Secretary  of  Defense 

PAUC 

procurement  average  unit  cost 

R&D 

research  and  development 

RFP 

request  for  proposals 

SAR 

Selected  Acquisition  Report 

TECHEVAL 

technical  evaluation 

TPP 

toul  package  procurement 
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